-- Logs begin at Wed 2022-08-10 11:46:26 UTC, end at Wed 2022-08-10 13:36:12 UTC. -- Aug 10 12:21:53.198283 dv-cinder-851996-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 10 12:21:53.206907 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 10 12:21:53.208963 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 10 12:21:53 2022] *** Aug 10 12:21:53.208963 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 10 12:21:53.208963 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 10 12:21:53.208963 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: nodename: dv-cinder-851996-7-iscsi Aug 10 12:21:53.208963 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: machine: x86_64 Aug 10 12:21:53.208963 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: clock source: unix Aug 10 12:21:53.208963 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: pcre jit disabled Aug 10 12:21:53.208963 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: detected number of CPU cores: 4 Aug 10 12:21:53.208963 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: current working directory: / Aug 10 12:21:53.208963 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: detected binary path: /usr/bin/uwsgi-core Aug 10 12:21:53.208963 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: your processes number limit is 47680 Aug 10 12:21:53.210395 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: your memory page size is 4096 bytes Aug 10 12:21:53.210395 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: detected max file descriptor number: 2048 Aug 10 12:21:53.210395 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: lock engine: pthread robust mutexes Aug 10 12:21:53.210395 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: thunder lock: enabled Aug 10 12:21:53.210395 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 10 12:21:53.210395 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 10 12:21:53.248640 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: Python main interpreter initialized at 0x55a0599ab9c0 Aug 10 12:21:53.248640 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: python threads support enabled Aug 10 12:21:53.248640 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: your server socket listen backlog is limited to 100 connections Aug 10 12:21:53.248640 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: your mercy for graceful operations on workers is 90 seconds Aug 10 12:21:53.248984 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: mapped 403077 bytes (393 KB) for 2 cores Aug 10 12:21:53.249059 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: *** Operational MODE: preforking *** Aug 10 12:21:53.249323 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: *** uWSGI is running in multiple interpreter mode *** Aug 10 12:21:53.249323 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: spawned uWSGI master process (pid: 80193) Aug 10 12:21:53.249521 dv-cinder-851996-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 10 12:21:53.253231 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: spawned uWSGI worker 1 (pid: 80202, cores: 1) Aug 10 12:21:53.253231 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: spawned uWSGI worker 2 (pid: 80203, cores: 1) Aug 10 12:21:53.253231 dv-cinder-851996-7-iscsi devstack@keystone.service[80193]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 10 12:21:58.775758 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 12:21:58.775758 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: from cryptography.utils import int_from_bytes Aug 10 12:21:58.776506 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 12:21:58.776506 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: from cryptography.utils import int_from_bytes Aug 10 12:21:58.776825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 12:21:58.776825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: from cryptography.utils import int_from_bytes Aug 10 12:21:58.777273 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 12:21:58.777273 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: from cryptography.utils import int_from_bytes Aug 10 12:21:58.826390 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 10 12:21:58.826570 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 10 12:21:58.826658 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 10 12:21:58.826754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 10 12:21:58.826834 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] command line args: [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 10 12:21:58.826906 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] command line args: [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 10 12:21:58.827012 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 10 12:21:58.827128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 10 12:21:58.827225 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ================================================================================ {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 10 12:21:58.827337 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ================================================================================ {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 10 12:21:58.827458 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] admin_token = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.827653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] admin_token = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.827737 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] config_dir = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.827880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.827964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] config_dir = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.828050 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] config_source = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.828158 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.828256 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] debug = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.828335 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] config_source = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.828406 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.828702 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] debug = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.828702 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.828848 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.828993 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.829102 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.829210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.829316 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.829411 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.829505 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.829646 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.829782 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.829903 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_config_append = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.829999 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.830092 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] list_limit = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.830191 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_dir = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.830292 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] log_config_append = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.830385 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_file = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.830487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.830588 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.830703 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] log_dir = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.830785 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] log_file = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.830905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.831010 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.831095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.831285 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.831369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.831486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.831628 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.831714 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.831877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.831969 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.832129 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.832234 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.832360 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.832445 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.832584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.832693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.832826 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.832931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.833021 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.833225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.833301 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.833500 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.833582 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.833777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.833876 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.834067 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.834269 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.834925 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.835049 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] publish_errors = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.835227 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.835325 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.835467 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.835649 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.835725 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.835899 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.836133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.836369 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] standard_threads = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.836451 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.836599 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.836702 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] publish_errors = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.836843 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.836922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.837020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.837182 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.837259 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] use_journal = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.837357 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] use_json = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.837439 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.837516 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] use_stderr = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.837686 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.837775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] use_syslog = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.837913 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.837990 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.838165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.838249 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] standard_threads = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.838400 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.838542 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.838673 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.838868 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.838949 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.839051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.839129 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.839325 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] use_journal = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.839413 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.839506 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] use_json = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.839669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.839746 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] use_stderr = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.839850 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] use_syslog = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.839959 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.840144 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:21:58.840315 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.840459 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.840664 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.840754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.840993 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.841072 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.841306 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.841433 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.841617 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.841707 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.841915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.841999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.842226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.842304 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.842529 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.842615 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.842829 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.842930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.843149 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.843282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.843479 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.843617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.843805 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.844014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.844121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.844377 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.844459 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.844733 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.844841 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.845086 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.845193 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.845352 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.845532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.845625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.845828 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.845912 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.846091 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.846166 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.846343 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.846498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.846572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.846732 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.846848 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.847071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.847154 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.847358 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.847443 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.847614 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.847692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.847852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.847929 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.848089 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.848167 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.848329 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.848405 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.848564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.848646 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.848815 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.848892 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.849045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.849128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.849284 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.849361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.849515 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.849598 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.849742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.849823 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.850083 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.850694 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.850872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.850977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.851258 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.851359 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.851491 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.851622 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.851699 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.851814 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.851889 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.852011 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.852082 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.852196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.852309 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.852384 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.852502 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.852584 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.852712 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.852801 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.852961 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.853082 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.853195 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.853275 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.853432 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.853510 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.853671 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.853762 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.853904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.853978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.854140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.854220 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.854375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.854461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.854604 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.854725 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.854845 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.854925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.855077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.855156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.855337 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.855426 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.855568 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.855653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.855803 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.855887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.856041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.856116 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.856269 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.856346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.856507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.856594 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.856742 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.856822 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.856980 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.857059 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.857217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.857297 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.857469 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.857547 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.857705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.857788 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.857939 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.858025 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.858176 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.858253 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.858412 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.858491 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.858660 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.858851 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.858929 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.859000 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.859122 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.859220 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.859391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.859486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.859628 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.859705 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.859861 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.859944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.860098 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.860191 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.860330 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.860409 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.860565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.860650 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.860807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.860884 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.861052 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.861130 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.861290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.861410 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.861523 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.861600 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.861757 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.861836 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.861989 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.862066 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.862224 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.862300 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.862455 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.862539 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.862691 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.862770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.862919 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.862997 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.863150 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.863304 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.863409 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.863489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.863640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.863720 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.863878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.863993 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.864097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.864175 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.864337 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.864414 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.864568 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.864647 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.864798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.864874 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.865037 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.865116 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.865273 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.865352 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.865518 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.866158 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.external = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.866158 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.866293 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.866293 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.866433 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.866505 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.866627 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.866627 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.password = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.866807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.866891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.token = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.866963 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.867067 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.867148 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.867258 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.867340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] auth.external = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.867412 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.867483 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.867649 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.867732 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.867900 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.867980 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.868148 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.868231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] auth.password = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.868380 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.868460 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] auth.token = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.868624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.868749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.868860 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.868964 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.869128 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.869255 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.869381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.869507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.869640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.869786 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.870162 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.870290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.870416 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.870542 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.870675 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.870813 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] credential.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.870942 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.871069 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.871288 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.871370 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.871443 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.871523 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.871523 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.871653 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.871725 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.871796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.871943 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.872030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.872214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.872293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.872470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.872548 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.872708 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.872864 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.872941 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.873012 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.873177 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.873257 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.873425 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.873513 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.873658 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.873737 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.873906 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.873985 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.874143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.874231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.874398 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.874477 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.874634 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.874711 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.874884 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.874975 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.875118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.875215 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.875381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.875459 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.875615 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.875706 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.875862 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.875941 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.876107 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.876186 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.876343 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.876430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.876577 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.876656 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.876825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.876904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.877053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.877147 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.877291 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.877370 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.877539 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.877616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.877788 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.877874 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.878020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.878100 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.878251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.878329 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.878483 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.878571 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.878715 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.878793 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.878956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.879034 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.879221 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.879307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.879479 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.879559 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.879711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.879799 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.879944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.880025 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.880177 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.880256 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.880425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.880504 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.880664 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.880742 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.880902 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.880984 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.881133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.881242 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.881431 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.881511 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.881677 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.881760 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.881918 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.882000 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.882165 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.882245 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.882393 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.882472 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.882631 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.882709 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.882867 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.882947 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.883106 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.883211 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.883373 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.883453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.883617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.883697 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.883846 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.883924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.884084 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.884163 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.884313 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.884393 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.884552 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.884631 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.884788 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.884867 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.885031 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.885110 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.885268 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.885346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.885497 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.885575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.885739 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.885819 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.885972 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.886051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.886202 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.886280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.886434 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.886514 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.886674 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.886751 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.886914 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.886993 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.887149 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.887289 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.887411 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.887492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.887649 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.887729 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.887880 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.887960 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.888118 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.888197 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.888374 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.888452 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.888606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.888687 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.888845 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.888923 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.889085 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.889164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.889323 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.889404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.889554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.889632 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.889785 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.889864 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.890021 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.890108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.890256 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.890335 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.890492 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.890571 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.890722 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.890802 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.890968 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.891047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.891222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.891317 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.891461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.891541 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.891701 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.891780 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.891939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.892027 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.892167 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.892245 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.892399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.892477 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.892641 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.892728 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.892876 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.892956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.893111 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.893189 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.893343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.893431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.893588 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.893668 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.893818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.893895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.894058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.894148 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.894293 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.894371 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.894528 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.894606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.894763 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.894849 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.894995 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.895074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.895258 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.895338 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.895496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.895647 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.895723 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.895800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.895972 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.896051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.896202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.896292 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.896435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.896512 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.896671 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.896748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.897453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.897545 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.897704 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.897790 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.897945 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.898060 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.898190 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.898279 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.898440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.898520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.898676 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.898762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.898924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.899011 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.899157 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.899305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.899418 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.899498 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.899652 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] role.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.899744 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.899900 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] role.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.899980 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.900129 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.900208 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.900375 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.900462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.900620 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.900699 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.900856 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.900934 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.901099 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.901186 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.901339 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] role.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.901417 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.901583 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.901661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] role.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.901813 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.901899 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.902064 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.902144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.902293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.902372 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.902540 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.902627 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.902787 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.902866 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.903037 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.903117 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.903313 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.903397 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.903566 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.903707 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.903817 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.903943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.904051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.904195 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.904303 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.904445 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.904553 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.904799 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.904888 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.905047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.905132 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.905279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.905444 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.905524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.905683 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.905763 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.905938 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.906017 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.906191 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.906270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.906435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.906515 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.906667 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.906753 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.906904 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.906981 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.907161 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.907289 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.907432 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.907548 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.907673 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.907796 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.907911 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.908038 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.908142 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.908276 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.908378 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.908515 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.908603 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.908778 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.908857 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.909039 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.909118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.909293 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.909381 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.909538 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.909617 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.909780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.909859 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.910016 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.910104 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.910255 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.910333 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.910493 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.910571 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.910727 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.910806 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.910966 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.911045 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.911219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.911302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.911467 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.911548 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.911717 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.911797 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.911948 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.912027 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.912190 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.912271 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.912481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.913097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.913262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.913349 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.connection = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.913505 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.913595 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.913757 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.913844 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.914010 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.914090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.914265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.914344 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.914499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.914579 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.914733 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.914812 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.914972 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.915051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.915229 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.915314 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.connection = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.915477 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.915567 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.915713 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.915792 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.915955 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.916110 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.916186 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.916265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.916426 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.916506 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.916661 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.916749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.916904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.916983 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.917134 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.917213 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.917374 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.917461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.917608 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.917687 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.917779 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 10 12:21:58.917859 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.918076 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.918329 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.918569 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.918822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.919058 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.919338 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.919580 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:21:58.919754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 10 12:21:58.946854 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:58.947076 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:58.947331 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:58.947974 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:58.953522 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:58.953522 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:58.958121 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:58.963517 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:58.987591 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:58.987823 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:58.989898 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:58.990128 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.043441 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.043605 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.094332 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.094555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.100886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.101123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.159679 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.159893 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.159998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.160113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.160196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.160334 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.160416 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.160551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.160635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.160763 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.160844 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.160977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.184834 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.184965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.185131 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.185225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.185301 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.185421 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.185506 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.185648 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.185731 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.185866 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.185949 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.186079 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.186163 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.186293 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.197795 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.198143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.215601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.215834 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.215923 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.216090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.216175 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.216315 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.216417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.216532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.216616 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.216745 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.216828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.216957 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.217042 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.217168 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:21:59.234451 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.234658 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 12:21:59.234872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.235003 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.235087 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 12:21:59.235218 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 12:21:59.235448 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.235532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.235649 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 12:21:59.235733 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.235926 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.236008 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.236191 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.236282 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.236453 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.236713 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.248440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.248719 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.248991 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.249089 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.249275 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.249360 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.249622 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.249913 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.256202 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.257215 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.259322 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.259613 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.259896 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.260179 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.260326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.260460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.260616 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.260722 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.260915 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.261008 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.261205 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.261287 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.261500 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.261582 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.261771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.261854 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.262041 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.262124 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.262317 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.262595 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.262879 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.263167 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.284871 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.286354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.286457 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.286641 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.288460 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.288755 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.291826 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.292114 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.292399 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.293363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.293694 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.293940 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.298942 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.299257 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.300516 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.300809 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.303921 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.304215 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.304504 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.304807 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.305088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.305383 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.305492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.305667 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.305790 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.305953 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.306077 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.306365 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.306644 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.306926 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.307224 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.307501 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.321891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.322174 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.323188 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.323478 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.326092 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.327473 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.329238 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.329535 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.330625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.330924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.333897 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.335373 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.337198 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.338538 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.340372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.340676 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.340961 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.341250 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.341528 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.341715 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.341823 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.342019 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.342307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.342604 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.342886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.343168 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.358181 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.358473 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.358744 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.359019 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.359611 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.359914 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.360190 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.360458 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.366404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.368055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.368205 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.368336 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.369765 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.370298 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.371415 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.371702 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.372005 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.372294 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.372571 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.373151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.373439 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.373731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.374022 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.374302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.383989 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.384273 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.384554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.384836 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.385120 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.385399 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.385677 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.385837 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.386124 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.386410 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.386696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.386984 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.387292 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.387573 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.402512 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.404002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.405470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.406962 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.408405 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.409913 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.410055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.411479 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.411609 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.411767 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.412065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.413617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.413909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.414196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.419085 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.420670 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.420811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.422283 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.423611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.423901 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.424185 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.424464 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.425230 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.425526 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.425822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.426107 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.433314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.433605 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.433888 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.434777 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.435073 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.435402 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.440848 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.441138 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.441423 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.441700 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.441973 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.442214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:21:59.442343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.442497 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.442627 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.442776 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.442889 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.443095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.443242 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.443369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.443498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.443640 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.443877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.444009 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.444144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.444274 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.444411 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.444542 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.444671 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.444823 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.445102 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.445372 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.445660 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.445927 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:21:59.477934 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55a0599ab9c0 pid: 80203 (default app) Aug 10 12:21:59.479878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55a0599ab9c0 pid: 80202 (default app) Aug 10 12:21:59.481202 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217c6d5-d1e0-4bff-89fa-14d831bb2828 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:59.481490 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217c6d5-d1e0-4bff-89fa-14d831bb2828 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:59.481759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217c6d5-d1e0-4bff-89fa-14d831bb2828 None None] PATH_INFO: `/v3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:59.486022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1/1] 10.222.0.49 () {50 vars in 791 bytes} [Wed Aug 10 12:21:59 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:07.969525 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827cd5c-4a4d-4ee7-94c1-a760e99b5fb9 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:07.970113 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827cd5c-4a4d-4ee7-94c1-a760e99b5fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:07.970113 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827cd5c-4a4d-4ee7-94c1-a760e99b5fb9 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:07.974635 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1/2] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:07 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:22:07.980462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bbdf8b-22bb-428f-b99d-43812dd24a52 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:07.980888 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bbdf8b-22bb-428f-b99d-43812dd24a52 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:07.980991 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bbdf8b-22bb-428f-b99d-43812dd24a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:08.052806 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: INFO dbcounter [None req-95bbdf8b-22bb-428f-b99d-43812dd24a52 None None] Registered counter for database keystone Aug 10 12:22:08.072055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG oslo_db.sqlalchemy.engines [None req-95bbdf8b-22bb-428f-b99d-43812dd24a52 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80203) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 10 12:22:08.078356 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writer thread running {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 10 12:22:08.259622 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG passlib.handlers.bcrypt [None req-95bbdf8b-22bb-428f-b99d-43812dd24a52 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80203) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 10 12:22:08.260047 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG passlib.handlers.bcrypt [None req-95bbdf8b-22bb-428f-b99d-43812dd24a52 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80203) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 10 12:22:08.315993 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-95bbdf8b-22bb-428f-b99d-43812dd24a52 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:08.317624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-95bbdf8b-22bb-428f-b99d-43812dd24a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:08.448994 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 2/3] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:07 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 10 12:22:08.586040 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827cd5c-4a4d-4ee7-94c1-a760e99b5fb9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:08.656525 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: INFO dbcounter [None req-5827cd5c-4a4d-4ee7-94c1-a760e99b5fb9 None None] Registered counter for database keystone Aug 10 12:22:08.677914 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG oslo_db.sqlalchemy.engines [None req-5827cd5c-4a4d-4ee7-94c1-a760e99b5fb9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80202) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 10 12:22:08.684917 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writer thread running {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 10 12:22:08.806699 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827cd5c-4a4d-4ee7-94c1-a760e99b5fb9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:08.811055 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d8a55-e054-4935-9331-de8d0753a925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:08.811716 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d8a55-e054-4935-9331-de8d0753a925 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:08.812003 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d8a55-e054-4935-9331-de8d0753a925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:08.812309 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d8a55-e054-4935-9331-de8d0753a925 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:08.821133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d8a55-e054-4935-9331-de8d0753a925 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:08.875077 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7c6d8a55-e054-4935-9331-de8d0753a925 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 10 12:22:08.875399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d8a55-e054-4935-9331-de8d0753a925 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:08.885579 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 2/4] 10.222.0.49 () {58 vars in 1324 bytes} [Wed Aug 10 12:22:08 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 12:22:08.891471 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bbdf8b-22bb-428f-b99d-43812dd24a52 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:08.905751 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bbdf8b-22bb-428f-b99d-43812dd24a52 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:08.910216 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf90c61-371f-4c00-961e-f7df5a8875ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:08.911356 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf90c61-371f-4c00-961e-f7df5a8875ee admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:08.911645 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf90c61-371f-4c00-961e-f7df5a8875ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:08.911919 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf90c61-371f-4c00-961e-f7df5a8875ee admin admin] PATH_INFO: `/v3/projects/3c65e78604254a1dbcd377669ee26423` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:08.913518 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf90c61-371f-4c00-961e-f7df5a8875ee admin admin] RBAC: Authorizing `identity:get_project(project_id=3c65e78604254a1dbcd377669ee26423)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:08.965666 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7bf90c61-371f-4c00-961e-f7df5a8875ee admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 10 12:22:08.966017 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf90c61-371f-4c00-961e-f7df5a8875ee admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:08.970599 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 3/5] 10.222.0.49 () {56 vars in 1207 bytes} [Wed Aug 10 12:22:08 2022] GET /identity/v3/projects/3c65e78604254a1dbcd377669ee26423 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:10.932664 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba5f2a-4c91-49a1-bcb8-76e8d0171cc9 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:10.933197 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba5f2a-4c91-49a1-bcb8-76e8d0171cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:10.933197 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba5f2a-4c91-49a1-bcb8-76e8d0171cc9 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:10.937329 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 3/6] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:22:10.947211 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78fb9e-53b5-4d98-b104-2c1347338c25 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:10.947211 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78fb9e-53b5-4d98-b104-2c1347338c25 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:10.947211 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78fb9e-53b5-4d98-b104-2c1347338c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:10.965691 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4b78fb9e-53b5-4d98-b104-2c1347338c25 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:10.967245 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4b78fb9e-53b5-4d98-b104-2c1347338c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:10.975347 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 4/7] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 10 12:22:10.997666 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba5f2a-4c91-49a1-bcb8-76e8d0171cc9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:11.010264 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba5f2a-4c91-49a1-bcb8-76e8d0171cc9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:11.014431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a1ee4-bab7-432e-993b-c5a71a42f529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:11.015021 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a1ee4-bab7-432e-993b-c5a71a42f529 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:11.015341 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a1ee4-bab7-432e-993b-c5a71a42f529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:11.015619 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a1ee4-bab7-432e-993b-c5a71a42f529 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:11.023251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19a1ee4-bab7-432e-993b-c5a71a42f529 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:11.025006 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19a1ee4-bab7-432e-993b-c5a71a42f529 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:11.033608 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 4/8] 10.222.0.49 () {58 vars in 1324 bytes} [Wed Aug 10 12:22:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 12:22:11.039310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78fb9e-53b5-4d98-b104-2c1347338c25 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:11.048262 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78fb9e-53b5-4d98-b104-2c1347338c25 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:11.052565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9202b-0b8e-4501-a69c-003e535cfe9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:11.053692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9202b-0b8e-4501-a69c-003e535cfe9c admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:11.053964 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9202b-0b8e-4501-a69c-003e535cfe9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:11.054239 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9202b-0b8e-4501-a69c-003e535cfe9c admin admin] PATH_INFO: `/v3/users/903d09eed7334b43af767deea1a6bd32` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:11.055575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9202b-0b8e-4501-a69c-003e535cfe9c admin admin] RBAC: Authorizing `identity:get_user(user_id=903d09eed7334b43af767deea1a6bd32)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:11.057376 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9202b-0b8e-4501-a69c-003e535cfe9c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:11.058868 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 5/9] 10.222.0.49 () {56 vars in 1198 bytes} [Wed Aug 10 12:22:11 2022] GET /identity/v3/users/903d09eed7334b43af767deea1a6bd32 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:13.134304 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215f573-f2f0-4ba8-83b3-2ea7a5da171b None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:13.134844 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215f573-f2f0-4ba8-83b3-2ea7a5da171b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:13.134940 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215f573-f2f0-4ba8-83b3-2ea7a5da171b None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:13.136281 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 5/10] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:13.139525 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd14a9-5eb8-40ed-9b19-b15583fec1a2 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:13.139934 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd14a9-5eb8-40ed-9b19-b15583fec1a2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:13.140083 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd14a9-5eb8-40ed-9b19-b15583fec1a2 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:13.141383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3ba72-1e2a-4c2f-889e-f3a551e779e0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:13.141562 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 6/11] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:22:13.141732 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3ba72-1e2a-4c2f-889e-f3a551e779e0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:13.142014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3ba72-1e2a-4c2f-889e-f3a551e779e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:13.146345 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7447d22-5f29-4e9d-92d9-bf8c2ae3c79a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:13.146628 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7447d22-5f29-4e9d-92d9-bf8c2ae3c79a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:13.147011 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7447d22-5f29-4e9d-92d9-bf8c2ae3c79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:13.164685 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b7447d22-5f29-4e9d-92d9-bf8c2ae3c79a None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:13.165850 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b7447d22-5f29-4e9d-92d9-bf8c2ae3c79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:13.185138 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG passlib.handlers.bcrypt [None req-93e3ba72-1e2a-4c2f-889e-f3a551e779e0 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80202) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 10 12:22:13.185552 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG passlib.handlers.bcrypt [None req-93e3ba72-1e2a-4c2f-889e-f3a551e779e0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80202) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 10 12:22:13.230928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-93e3ba72-1e2a-4c2f-889e-f3a551e779e0 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:13.232583 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-93e3ba72-1e2a-4c2f-889e-f3a551e779e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:13.243971 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 7/12] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:22:13.266536 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7447d22-5f29-4e9d-92d9-bf8c2ae3c79a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:13.292658 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:13.336733 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 6/13] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:13.359234 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3ba72-1e2a-4c2f-889e-f3a551e779e0 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:13.373667 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7447d22-5f29-4e9d-92d9-bf8c2ae3c79a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:13.409270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572854f6-b373-4845-93e7-5bbe3269d7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:13.409832 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572854f6-b373-4845-93e7-5bbe3269d7db None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:13.410116 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572854f6-b373-4845-93e7-5bbe3269d7db None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:13.410395 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572854f6-b373-4845-93e7-5bbe3269d7db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:13.459857 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3ba72-1e2a-4c2f-889e-f3a551e779e0 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:13.462102 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:13.477508 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572854f6-b373-4845-93e7-5bbe3269d7db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:13.480826 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572854f6-b373-4845-93e7-5bbe3269d7db None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:13.498387 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fed778-ffcb-42dc-a928-3ffaa01fe605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:13.499590 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fed778-ffcb-42dc-a928-3ffaa01fe605 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:13.500123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fed778-ffcb-42dc-a928-3ffaa01fe605 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:13.500460 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fed778-ffcb-42dc-a928-3ffaa01fe605 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:13.502979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:13.513115 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fed778-ffcb-42dc-a928-3ffaa01fe605 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:13.521517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-d6fed778-ffcb-42dc-a928-3ffaa01fe605 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:22:13.521517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:13.521517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:13.521517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:13.521517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:13.521517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:13.521517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:13.521517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:13.521517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:13.521517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:13.521517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:13.521517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:13.521517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:13.521517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:13.521517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:13.521517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:13.522639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:13.522639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:13.522639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:13.522639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:13.522639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:13.522639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:13.522639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:13.522639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:13.522639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:13.522639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:13.522639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:13.522639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:13.522639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:13.522639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:13.522639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:13.522639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:13.523683 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:13.523683 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:13.523683 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:13.523683 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:13.523683 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:13.523683 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:13.523683 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:13.523683 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:13.523683 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:13.523683 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:13.523683 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:22:13.523683 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:22:13.523683 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 7/14] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:22:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:13.530464 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fed778-ffcb-42dc-a928-3ffaa01fe605 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:13.582012 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 8/15] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:22:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 12:22:13.587793 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572854f6-b373-4845-93e7-5bbe3269d7db None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:13.631639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fed778-ffcb-42dc-a928-3ffaa01fe605 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:13.664753 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5426cc8b-059a-4ab5-86f4-4e71a583a6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:13.665311 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426cc8b-059a-4ab5-86f4-4e71a583a6e6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:13.665593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426cc8b-059a-4ab5-86f4-4e71a583a6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:13.665864 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426cc8b-059a-4ab5-86f4-4e71a583a6e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:13.666549 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5426cc8b-059a-4ab5-86f4-4e71a583a6e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:13.666853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5426cc8b-059a-4ab5-86f4-4e71a583a6e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:13.668367 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5426cc8b-059a-4ab5-86f4-4e71a583a6e6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:13.684323 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 8/16] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:13.690194 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5426cc8b-059a-4ab5-86f4-4e71a583a6e6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:13.719743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:13.795527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5426cc8b-059a-4ab5-86f4-4e71a583a6e6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:13.796125 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572854f6-b373-4845-93e7-5bbe3269d7db None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:13.819946 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:13.829775 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d12cc9-e879-4d40-84dc-c2292822062b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:13.830411 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d12cc9-e879-4d40-84dc-c2292822062b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:13.830689 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d12cc9-e879-4d40-84dc-c2292822062b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:13.830961 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d12cc9-e879-4d40-84dc-c2292822062b None admin] PATH_INFO: `/v3/users/903d09eed7334b43af767deea1a6bd32` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:13.831195 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59da891-1db7-45ec-8616-d5e2dbfe1639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:13.831848 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59da891-1db7-45ec-8616-d5e2dbfe1639 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:13.832131 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59da891-1db7-45ec-8616-d5e2dbfe1639 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:13.832409 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59da891-1db7-45ec-8616-d5e2dbfe1639 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:13.832503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d12cc9-e879-4d40-84dc-c2292822062b None admin] RBAC: Authorizing `identity:get_user(user_id=903d09eed7334b43af767deea1a6bd32)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:13.833915 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d12cc9-e879-4d40-84dc-c2292822062b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:13.835679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 9/17] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:13 2022] GET /identity/v3/users/903d09eed7334b43af767deea1a6bd32 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:13.841134 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d12cc9-e879-4d40-84dc-c2292822062b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:13.842573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59da891-1db7-45ec-8616-d5e2dbfe1639 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:13.844164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59da891-1db7-45ec-8616-d5e2dbfe1639 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:13.862654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b59da891-1db7-45ec-8616-d5e2dbfe1639 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:22:13.862654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:13.862654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:13.862654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:13.862654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:13.862654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:13.862654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:13.862654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:13.862654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:13.862654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:13.862654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:13.862654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:13.862654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:22:13.862654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:22:13.862654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:22:13.862654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:22:13.863610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:13.863610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:13.863610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:13.863610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:13.863610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:13.863610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:13.863610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:13.863610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:13.863610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:13.863610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:13.863610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:13.863610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:13.863610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:13.863610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:13.863610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:13.863610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:13.863610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:22:13.864413 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:22:13.864413 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:22:13.864413 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:22:13.864413 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 9/18] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:22:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:22:13.870424 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59da891-1db7-45ec-8616-d5e2dbfe1639 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:13.934626 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:13.941285 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d12cc9-e879-4d40-84dc-c2292822062b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:13.970695 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59da891-1db7-45ec-8616-d5e2dbfe1639 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:13.975933 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b46288-9f27-4513-9c13-df3075b36aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:13.976602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b46288-9f27-4513-9c13-df3075b36aae None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:13.976886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b46288-9f27-4513-9c13-df3075b36aae None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:13.977163 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b46288-9f27-4513-9c13-df3075b36aae None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:13.978292 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b46288-9f27-4513-9c13-df3075b36aae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:13.979862 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b46288-9f27-4513-9c13-df3075b36aae None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:13.981584 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 10/19] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:22:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:13.987231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b46288-9f27-4513-9c13-df3075b36aae None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:14.007937 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0da24c-7415-410c-84fe-d6950c0a9f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:14.008551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0da24c-7415-410c-84fe-d6950c0a9f8f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:14.008834 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0da24c-7415-410c-84fe-d6950c0a9f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:14.009110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0da24c-7415-410c-84fe-d6950c0a9f8f None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:14.009762 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0da24c-7415-410c-84fe-d6950c0a9f8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:14.010071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0da24c-7415-410c-84fe-d6950c0a9f8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:14.011482 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0da24c-7415-410c-84fe-d6950c0a9f8f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:14.028272 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 10/20] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:14.091753 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b46288-9f27-4513-9c13-df3075b36aae None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:14.127854 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f57daa-fd56-48bd-b20e-6486b8f75061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:14.128482 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f57daa-fd56-48bd-b20e-6486b8f75061 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:14.128767 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f57daa-fd56-48bd-b20e-6486b8f75061 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:14.129045 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f57daa-fd56-48bd-b20e-6486b8f75061 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:14.129782 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f57daa-fd56-48bd-b20e-6486b8f75061 None admin] RBAC: Adding query filter params (role.id=bffbe227cda1418cbb58394ddff6f861, scope.domain.id=default, user.id=903d09eed7334b43af767deea1a6bd32) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:14.130096 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f57daa-fd56-48bd-b20e-6486b8f75061 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:14.132315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f57daa-fd56-48bd-b20e-6486b8f75061 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:14.141483 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 11/21] 10.222.0.49 () {56 vars in 1322 bytes} [Wed Aug 10 12:22:13 2022] GET /identity/v3/role_assignments?user.id=903d09eed7334b43af767deea1a6bd32&scope.domain.id=default&role.id=bffbe227cda1418cbb58394ddff6f861 => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:16.308127 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c765de-608b-4afe-b148-358abc8785aa None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:16.309099 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c765de-608b-4afe-b148-358abc8785aa None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:16.309604 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c765de-608b-4afe-b148-358abc8785aa None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:16.311433 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 11/22] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:16.317083 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2a2d0-77be-49d6-97e6-bf780a4b7172 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:16.317524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2a2d0-77be-49d6-97e6-bf780a4b7172 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:16.317709 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2a2d0-77be-49d6-97e6-bf780a4b7172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:16.344914 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c5b2a2d0-77be-49d6-97e6-bf780a4b7172 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:16.346150 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c5b2a2d0-77be-49d6-97e6-bf780a4b7172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:16.371450 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:16.432901 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 12/23] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:16.465264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c765de-608b-4afe-b148-358abc8785aa None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:16.525270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:16.609611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c765de-608b-4afe-b148-358abc8785aa None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:16.622997 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a99804-9b88-4708-8228-10ff64a170ac None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:16.623523 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a99804-9b88-4708-8228-10ff64a170ac None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:16.623952 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a99804-9b88-4708-8228-10ff64a170ac None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:16.625463 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 13/24] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:22:16.631045 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd3e99-8af1-4656-be7c-ee862c55bf46 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:16.632194 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd3e99-8af1-4656-be7c-ee862c55bf46 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:16.632622 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd3e99-8af1-4656-be7c-ee862c55bf46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:16.671210 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a4f15-66b0-46bb-bd26-751e1895582d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:16.671210 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a4f15-66b0-46bb-bd26-751e1895582d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:16.671210 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a4f15-66b0-46bb-bd26-751e1895582d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:16.671210 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a4f15-66b0-46bb-bd26-751e1895582d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:16.672657 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9afd3e99-8af1-4656-be7c-ee862c55bf46 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:16.674020 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9afd3e99-8af1-4656-be7c-ee862c55bf46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:16.691212 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a4f15-66b0-46bb-bd26-751e1895582d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:16.691212 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a4f15-66b0-46bb-bd26-751e1895582d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:16.704435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-4c6a4f15-66b0-46bb-bd26-751e1895582d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:22:16.704435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:16.704435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:16.704435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:16.704435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:16.704435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:16.704435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:16.704435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:16.704435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:16.704435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:16.704435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:16.704435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:16.704435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:22:16.704435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:22:16.704435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:22:16.704435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:22:16.705491 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:16.705491 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:16.705491 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:16.705491 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:16.705491 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:16.705491 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:16.705491 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:16.705491 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:16.705491 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:16.705491 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:16.705491 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:16.705491 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:16.705491 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:16.705491 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:16.705491 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:16.705491 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:16.705491 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:22:16.706406 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:22:16.706406 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:22:16.706406 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:22:16.708142 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 12/25] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:22:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:16.710186 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83835db3-04ef-453e-9b77-811bdda1d173 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:16.710612 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83835db3-04ef-453e-9b77-811bdda1d173 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:16.711037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83835db3-04ef-453e-9b77-811bdda1d173 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:16.712423 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 13/26] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:16.713928 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83835db3-04ef-453e-9b77-811bdda1d173 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:16.811556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:16.823763 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83835db3-04ef-453e-9b77-811bdda1d173 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:16.861964 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b211271-7a53-45f9-824a-4306141f4084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:16.862860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b211271-7a53-45f9-824a-4306141f4084 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:16.863318 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b211271-7a53-45f9-824a-4306141f4084 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:16.863743 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b211271-7a53-45f9-824a-4306141f4084 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:16.864542 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b211271-7a53-45f9-824a-4306141f4084 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:16.865451 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b211271-7a53-45f9-824a-4306141f4084 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:16.867656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b211271-7a53-45f9-824a-4306141f4084 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:16.876504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 14/27] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:22:16.877989 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ac372-85aa-414c-a3e6-456b12c51e41 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:16.878427 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ac372-85aa-414c-a3e6-456b12c51e41 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:16.878846 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ac372-85aa-414c-a3e6-456b12c51e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:16.885947 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 14/28] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:16.892077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b211271-7a53-45f9-824a-4306141f4084 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:16.924223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-cd5ac372-85aa-414c-a3e6-456b12c51e41 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:16.925455 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cd5ac372-85aa-414c-a3e6-456b12c51e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:17.021274 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b211271-7a53-45f9-824a-4306141f4084 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:17.109128 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eaabfe-ddff-48db-89be-9fcd1b0e333a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:17.109778 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eaabfe-ddff-48db-89be-9fcd1b0e333a None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:17.110064 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eaabfe-ddff-48db-89be-9fcd1b0e333a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:17.110340 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eaabfe-ddff-48db-89be-9fcd1b0e333a None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:17.111018 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eaabfe-ddff-48db-89be-9fcd1b0e333a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:17.112678 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eaabfe-ddff-48db-89be-9fcd1b0e333a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:17.164282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 15/29] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:17.165124 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ac372-85aa-414c-a3e6-456b12c51e41 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:17.193676 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:17.238039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:17.288790 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ac372-85aa-414c-a3e6-456b12c51e41 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:17.327055 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9738327-95ee-4915-9986-0aeb66336b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:17.327823 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9738327-95ee-4915-9986-0aeb66336b46 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:17.328047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9738327-95ee-4915-9986-0aeb66336b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:17.328255 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9738327-95ee-4915-9986-0aeb66336b46 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:17.328930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9738327-95ee-4915-9986-0aeb66336b46 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:17.330309 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9738327-95ee-4915-9986-0aeb66336b46 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:17.374052 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 15/30] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 10 12:22:16 2022] POST /identity/v3/projects => generated 301 bytes in 482 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:22:17.376110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d65e7e-fdea-45ca-a9af-dc685d5a1971 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:17.376618 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d65e7e-fdea-45ca-a9af-dc685d5a1971 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:17.377098 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d65e7e-fdea-45ca-a9af-dc685d5a1971 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:17.382892 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 16/31] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:17.384519 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d65e7e-fdea-45ca-a9af-dc685d5a1971 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:17.566985 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d65e7e-fdea-45ca-a9af-dc685d5a1971 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:17.608430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe88140-10c3-460a-b858-8214cca70230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:17.609080 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe88140-10c3-460a-b858-8214cca70230 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:17.609362 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe88140-10c3-460a-b858-8214cca70230 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:17.609642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe88140-10c3-460a-b858-8214cca70230 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:17.625249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe88140-10c3-460a-b858-8214cca70230 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:17.626852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 16/32] 10.222.0.49 () {60 vars in 1131 bytes} [Wed Aug 10 12:22:17 2022] POST /identity/v3/roles => generated 219 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:22:17.628541 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bccb91-1c57-474e-b614-963a7f65b36b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:17.628838 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bccb91-1c57-474e-b614-963a7f65b36b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:17.629116 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bccb91-1c57-474e-b614-963a7f65b36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:17.632375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-cbe88140-10c3-460a-b858-8214cca70230 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:22:17.632375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:17.632375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:17.632375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:17.632375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:17.632375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:17.632375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:17.632375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:17.632375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:17.632375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:17.632375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:17.632375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:17.632375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:17.632375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:17.632375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:17.632375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:17.633279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:17.633279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:17.633279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:17.633279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:17.633279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:17.633279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:17.633279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:17.633279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:17.633279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:17.633279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:17.633279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:17.633279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:17.633279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:17.633279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:17.633279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:17.633279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:17.634033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:17.634033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:17.634033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:17.634033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:17.634033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:17.634033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:17.634033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:17.634033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:17.634033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:17.634033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:17.634033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:22:17.634033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:22:17.634033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 17/33] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:22:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:17.639619 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe88140-10c3-460a-b858-8214cca70230 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:17.652182 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-30bccb91-1c57-474e-b614-963a7f65b36b None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:17.654891 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-30bccb91-1c57-474e-b614-963a7f65b36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:17.716844 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:17.759900 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 17/34] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:17.789120 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bccb91-1c57-474e-b614-963a7f65b36b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:17.792722 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe88140-10c3-460a-b858-8214cca70230 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:17.799935 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:17.834296 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee286e-38bb-45ba-b1b0-1298fcd60ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:17.834925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee286e-38bb-45ba-b1b0-1298fcd60ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:17.835296 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee286e-38bb-45ba-b1b0-1298fcd60ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:17.835508 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee286e-38bb-45ba-b1b0-1298fcd60ae2 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:17.836198 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee286e-38bb-45ba-b1b0-1298fcd60ae2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:17.836503 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee286e-38bb-45ba-b1b0-1298fcd60ae2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:17.838078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee286e-38bb-45ba-b1b0-1298fcd60ae2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:17.852923 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 18/35] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:17.858647 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee286e-38bb-45ba-b1b0-1298fcd60ae2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:17.936014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bccb91-1c57-474e-b614-963a7f65b36b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:17.979784 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea52f0d-8a2e-49c0-8c62-e479ae25d00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:17.980431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea52f0d-8a2e-49c0-8c62-e479ae25d00a None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:17.980717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea52f0d-8a2e-49c0-8c62-e479ae25d00a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:17.980999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea52f0d-8a2e-49c0-8c62-e479ae25d00a None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:17.981729 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea52f0d-8a2e-49c0-8c62-e479ae25d00a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:17.984849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea52f0d-8a2e-49c0-8c62-e479ae25d00a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:17.993343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee286e-38bb-45ba-b1b0-1298fcd60ae2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:18.013996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:18.027547 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777862d-8d4a-4ed1-a15b-07f4a670381b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:18.028184 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777862d-8d4a-4ed1-a15b-07f4a670381b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:18.028469 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777862d-8d4a-4ed1-a15b-07f4a670381b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:18.028747 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777862d-8d4a-4ed1-a15b-07f4a670381b None admin] PATH_INFO: `/v3/users/903d09eed7334b43af767deea1a6bd32` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:18.029993 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777862d-8d4a-4ed1-a15b-07f4a670381b None admin] RBAC: Authorizing `identity:get_user(user_id=903d09eed7334b43af767deea1a6bd32)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:18.031502 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777862d-8d4a-4ed1-a15b-07f4a670381b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:18.038443 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 19/36] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:17 2022] GET /identity/v3/users/903d09eed7334b43af767deea1a6bd32 => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:18.038443 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ae657-ce15-49c3-af90-6e54f1dcc292 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:18.038443 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ae657-ce15-49c3-af90-6e54f1dcc292 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:18.038443 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ae657-ce15-49c3-af90-6e54f1dcc292 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:18.038443 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 20/37] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:18.040030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ae657-ce15-49c3-af90-6e54f1dcc292 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:18.107020 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 18/38] 10.222.0.49 () {60 vars in 1131 bytes} [Wed Aug 10 12:22:17 2022] POST /identity/v3/roles => generated 225 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:22:18.108109 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56767b6-3afc-4b60-a58d-75d8e7d6c0aa None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:18.108397 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56767b6-3afc-4b60-a58d-75d8e7d6c0aa None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:18.108677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56767b6-3afc-4b60-a58d-75d8e7d6c0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:18.128095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b56767b6-3afc-4b60-a58d-75d8e7d6c0aa None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:18.129504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b56767b6-3afc-4b60-a58d-75d8e7d6c0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:18.151214 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ae657-ce15-49c3-af90-6e54f1dcc292 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:18.191666 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3404b3e1-6c22-45cd-8522-786d2edc4d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:18.192229 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404b3e1-6c22-45cd-8522-786d2edc4d18 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:18.192512 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404b3e1-6c22-45cd-8522-786d2edc4d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:18.192787 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404b3e1-6c22-45cd-8522-786d2edc4d18 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:18.194634 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3404b3e1-6c22-45cd-8522-786d2edc4d18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:18.196163 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3404b3e1-6c22-45cd-8522-786d2edc4d18 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:18.202732 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 21/39] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:22:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:18.202732 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332f0b3-8f4c-442e-ac1b-fac5087cf6ac None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:18.202732 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332f0b3-8f4c-442e-ac1b-fac5087cf6ac None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:18.202732 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332f0b3-8f4c-442e-ac1b-fac5087cf6ac None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:18.202732 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 22/40] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:18.206164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332f0b3-8f4c-442e-ac1b-fac5087cf6ac None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:18.207509 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:18.224204 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 19/41] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:18.224204 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd374467-7ca3-4412-93ac-8f2fbad78104 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:18.224204 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd374467-7ca3-4412-93ac-8f2fbad78104 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:18.224204 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd374467-7ca3-4412-93ac-8f2fbad78104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:18.244004 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-dd374467-7ca3-4412-93ac-8f2fbad78104 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:18.246578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-dd374467-7ca3-4412-93ac-8f2fbad78104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:18.286328 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:18.330627 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 20/42] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:18.331619 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd374467-7ca3-4412-93ac-8f2fbad78104 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:18.342249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332f0b3-8f4c-442e-ac1b-fac5087cf6ac None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:18.377627 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e2558-3ebb-4b90-ad34-75a74cc91fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:18.378289 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e2558-3ebb-4b90-ad34-75a74cc91fd6 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:18.378565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e2558-3ebb-4b90-ad34-75a74cc91fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:18.378844 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e2558-3ebb-4b90-ad34-75a74cc91fd6 None admin] PATH_INFO: `/v3/domains/default/users/903d09eed7334b43af767deea1a6bd32/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:18.380793 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e2558-3ebb-4b90-ad34-75a74cc91fd6 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=903d09eed7334b43af767deea1a6bd32, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:18.382969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e2558-3ebb-4b90-ad34-75a74cc91fd6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:18.432179 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd374467-7ca3-4412-93ac-8f2fbad78104 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:18.457912 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:18.466703 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34fa2c-8029-48de-af23-52e8792af229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:18.467341 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34fa2c-8029-48de-af23-52e8792af229 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:18.467661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34fa2c-8029-48de-af23-52e8792af229 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:18.467939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34fa2c-8029-48de-af23-52e8792af229 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:18.468587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34fa2c-8029-48de-af23-52e8792af229 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:18.469978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34fa2c-8029-48de-af23-52e8792af229 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:18.474093 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 23/43] 10.222.0.49 () {58 vars in 1361 bytes} [Wed Aug 10 12:22:18 2022] PUT /identity/v3/domains/default/users/903d09eed7334b43af767deea1a6bd32/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:22:18.479354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e2558-3ebb-4b90-ad34-75a74cc91fd6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:18.562558 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 21/44] 10.222.0.49 () {60 vars in 1131 bytes} [Wed Aug 10 12:22:18 2022] POST /identity/v3/roles => generated 223 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:22:18.599346 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e2558-3ebb-4b90-ad34-75a74cc91fd6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:18.619438 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:18.643139 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b8cfc6-032f-4440-a970-ce341ea9e855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:18.643987 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8cfc6-032f-4440-a970-ce341ea9e855 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:18.644269 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8cfc6-032f-4440-a970-ce341ea9e855 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:18.644544 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8cfc6-032f-4440-a970-ce341ea9e855 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:18.646175 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b8cfc6-032f-4440-a970-ce341ea9e855 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:18.647789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b8cfc6-032f-4440-a970-ce341ea9e855 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:18.649440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 24/45] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:22:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:18.655571 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34fa2c-8029-48de-af23-52e8792af229 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:18.768603 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34fa2c-8029-48de-af23-52e8792af229 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:18.804108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aad967-afd3-44f3-b6f3-ecdbd71fac26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:18.804718 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aad967-afd3-44f3-b6f3-ecdbd71fac26 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:18.804995 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aad967-afd3-44f3-b6f3-ecdbd71fac26 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:18.805264 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aad967-afd3-44f3-b6f3-ecdbd71fac26 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:18.805897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aad967-afd3-44f3-b6f3-ecdbd71fac26 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:18.807363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aad967-afd3-44f3-b6f3-ecdbd71fac26 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:18.962578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 22/46] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 10 12:22:18 2022] POST /identity/v3/projects => generated 312 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:22:20.500293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b1647-c35b-4f87-ad25-1b779d7531e6 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:20.500820 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b1647-c35b-4f87-ad25-1b779d7531e6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:20.500820 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b1647-c35b-4f87-ad25-1b779d7531e6 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:20.502205 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 25/47] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:20.507972 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856681a7-857d-4838-8b3f-c970dc9bed93 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:20.508394 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856681a7-857d-4838-8b3f-c970dc9bed93 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:20.508528 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856681a7-857d-4838-8b3f-c970dc9bed93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:20.527997 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-856681a7-857d-4838-8b3f-c970dc9bed93 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:20.529251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-856681a7-857d-4838-8b3f-c970dc9bed93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:20.558922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:20.595221 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 23/48] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:20.618879 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b1647-c35b-4f87-ad25-1b779d7531e6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:20.720720 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b1647-c35b-4f87-ad25-1b779d7531e6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:20.754166 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c478f-cd10-4376-8b44-196590659088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:20.754795 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c478f-cd10-4376-8b44-196590659088 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:20.755081 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c478f-cd10-4376-8b44-196590659088 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:20.755390 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c478f-cd10-4376-8b44-196590659088 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:20.765468 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371c478f-cd10-4376-8b44-196590659088 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:20.770318 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-371c478f-cd10-4376-8b44-196590659088 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:22:20.770318 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:20.770318 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:20.770318 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:20.770318 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:20.770318 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:20.770318 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:20.770318 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:20.770318 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:20.770318 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:20.770318 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:20.770318 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:20.770318 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:20.770318 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:20.770318 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:20.770318 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:20.771133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:20.771133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:20.771133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:20.771133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:20.771133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:20.771133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:20.771133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:20.771133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:20.771133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:20.771133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:20.771133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:20.771133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:20.771133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:20.771133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:20.771133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:20.771133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:20.772668 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:20.772668 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:20.772668 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:20.772668 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:20.772668 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:20.772668 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:20.772668 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:20.772668 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:20.772668 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:20.772668 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:20.772668 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:22:20.772668 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:22:20.772668 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 26/49] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:22:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:20.778061 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856681a7-857d-4838-8b3f-c970dc9bed93 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:20.874557 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856681a7-857d-4838-8b3f-c970dc9bed93 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:20.907726 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d36ec9-cf67-4b8a-9da4-4805c8fee83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:20.908358 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d36ec9-cf67-4b8a-9da4-4805c8fee83e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:20.908617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d36ec9-cf67-4b8a-9da4-4805c8fee83e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:20.908890 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d36ec9-cf67-4b8a-9da4-4805c8fee83e None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:20.909547 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d36ec9-cf67-4b8a-9da4-4805c8fee83e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:20.909852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d36ec9-cf67-4b8a-9da4-4805c8fee83e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:20.911234 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d36ec9-cf67-4b8a-9da4-4805c8fee83e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:20.919871 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 24/50] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:20.925425 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c478f-cd10-4376-8b44-196590659088 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:20.963454 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958614d0-cd2e-454d-ae5d-7143c09e6c50 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:20.963740 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958614d0-cd2e-454d-ae5d-7143c09e6c50 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:20.964017 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958614d0-cd2e-454d-ae5d-7143c09e6c50 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:20.965363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 25/51] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:22:20.970343 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c37ad-bfea-4be6-b842-7beee19b0129 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:20.970636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c37ad-bfea-4be6-b842-7beee19b0129 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:20.970922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c37ad-bfea-4be6-b842-7beee19b0129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:20.973435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:20.989356 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5a2c37ad-bfea-4be6-b842-7beee19b0129 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:20.990734 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5a2c37ad-bfea-4be6-b842-7beee19b0129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:21.015131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:21.029349 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c478f-cd10-4376-8b44-196590659088 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:21.058361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 26/52] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:22:21.062272 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d864858-4d19-4600-b7be-4725d8209847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:21.062905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d864858-4d19-4600-b7be-4725d8209847 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:21.063200 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d864858-4d19-4600-b7be-4725d8209847 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:21.063486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d864858-4d19-4600-b7be-4725d8209847 None admin] PATH_INFO: `/v3/users/903d09eed7334b43af767deea1a6bd32` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:21.064602 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d864858-4d19-4600-b7be-4725d8209847 None admin] RBAC: Authorizing `identity:get_user(user_id=903d09eed7334b43af767deea1a6bd32)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:21.066045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d864858-4d19-4600-b7be-4725d8209847 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:21.067814 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 27/53] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:20 2022] GET /identity/v3/users/903d09eed7334b43af767deea1a6bd32 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:21.073684 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c37ad-bfea-4be6-b842-7beee19b0129 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:21.081287 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d864858-4d19-4600-b7be-4725d8209847 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:21.172566 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c37ad-bfea-4be6-b842-7beee19b0129 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:21.184641 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d864858-4d19-4600-b7be-4725d8209847 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:21.197135 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:21.206043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e175c27-28fc-4763-82b2-50cbc91eab72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:21.206684 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e175c27-28fc-4763-82b2-50cbc91eab72 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:21.206970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e175c27-28fc-4763-82b2-50cbc91eab72 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:21.207276 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e175c27-28fc-4763-82b2-50cbc91eab72 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:21.208396 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e175c27-28fc-4763-82b2-50cbc91eab72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:21.209858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e175c27-28fc-4763-82b2-50cbc91eab72 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:21.211677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 27/54] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:22:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:21.217209 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e175c27-28fc-4763-82b2-50cbc91eab72 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:21.217473 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b3bd8-43c3-431e-ba68-c61c5a1e6c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:21.220588 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b3bd8-43c3-431e-ba68-c61c5a1e6c6f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:21.220588 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b3bd8-43c3-431e-ba68-c61c5a1e6c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:21.220588 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b3bd8-43c3-431e-ba68-c61c5a1e6c6f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:21.220588 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b3bd8-43c3-431e-ba68-c61c5a1e6c6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:21.221319 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b3bd8-43c3-431e-ba68-c61c5a1e6c6f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:21.223043 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 28/55] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:22:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:21.229083 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b3bd8-43c3-431e-ba68-c61c5a1e6c6f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:21.247047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:21.319260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e175c27-28fc-4763-82b2-50cbc91eab72 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:21.329872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b3bd8-43c3-431e-ba68-c61c5a1e6c6f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:21.352552 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e3440-c4d1-4492-9398-10c85f72527a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:21.353159 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e3440-c4d1-4492-9398-10c85f72527a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:21.353437 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e3440-c4d1-4492-9398-10c85f72527a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:21.353710 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e3440-c4d1-4492-9398-10c85f72527a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:21.354426 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e3440-c4d1-4492-9398-10c85f72527a None admin] RBAC: Adding query filter params (role.id=bffbe227cda1418cbb58394ddff6f861, scope.domain.id=default, user.id=903d09eed7334b43af767deea1a6bd32) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:21.354733 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e3440-c4d1-4492-9398-10c85f72527a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:21.356205 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e3440-c4d1-4492-9398-10c85f72527a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:21.361814 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 28/56] 10.222.0.49 () {56 vars in 1322 bytes} [Wed Aug 10 12:22:21 2022] GET /identity/v3/role_assignments?user.id=903d09eed7334b43af767deea1a6bd32&scope.domain.id=default&role.id=bffbe227cda1418cbb58394ddff6f861 => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:21.363249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ac2f8-b809-4421-9be1-df3a41a12a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:21.363865 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ac2f8-b809-4421-9be1-df3a41a12a67 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:21.364150 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ac2f8-b809-4421-9be1-df3a41a12a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:21.364425 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ac2f8-b809-4421-9be1-df3a41a12a67 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:21.365070 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07ac2f8-b809-4421-9be1-df3a41a12a67 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:21.367071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07ac2f8-b809-4421-9be1-df3a41a12a67 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:21.442315 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 29/57] 10.222.0.49 () {60 vars in 1140 bytes} [Wed Aug 10 12:22:21 2022] POST /identity/v3/projects => generated 298 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:22:23.455061 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a059af-5ca3-448a-a05f-81fee0bca256 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:23.455655 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a059af-5ca3-448a-a05f-81fee0bca256 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:23.455655 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a059af-5ca3-448a-a05f-81fee0bca256 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:23.457007 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 29/58] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:23.462792 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635dd2e-ed35-4039-a79e-856a0c70f134 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:23.463229 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635dd2e-ed35-4039-a79e-856a0c70f134 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:23.463564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635dd2e-ed35-4039-a79e-856a0c70f134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:23.487141 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9635dd2e-ed35-4039-a79e-856a0c70f134 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:23.488711 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9635dd2e-ed35-4039-a79e-856a0c70f134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:23.518490 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:23.563525 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 30/59] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:23.588307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a059af-5ca3-448a-a05f-81fee0bca256 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:23.700298 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:23.710785 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a059af-5ca3-448a-a05f-81fee0bca256 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:23.748837 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19996b-2c10-497d-ab37-3669dcbd2af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:23.749475 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19996b-2c10-497d-ab37-3669dcbd2af1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:23.749758 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19996b-2c10-497d-ab37-3669dcbd2af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:23.750033 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19996b-2c10-497d-ab37-3669dcbd2af1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:23.751233 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19996b-2c10-497d-ab37-3669dcbd2af1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:23.752688 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19996b-2c10-497d-ab37-3669dcbd2af1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:23.754267 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 30/60] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:22:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:23.760657 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635dd2e-ed35-4039-a79e-856a0c70f134 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:23.861847 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635dd2e-ed35-4039-a79e-856a0c70f134 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:23.896689 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a2905-9157-4ad8-9642-15e7b12319ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:23.898003 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a2905-9157-4ad8-9642-15e7b12319ae None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:23.898288 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a2905-9157-4ad8-9642-15e7b12319ae None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:23.898568 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a2905-9157-4ad8-9642-15e7b12319ae None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:23.899253 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a2905-9157-4ad8-9642-15e7b12319ae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:23.900979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a2905-9157-4ad8-9642-15e7b12319ae None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:24.009638 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 31/61] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 12:22:23 2022] POST /identity/v3/users => generated 275 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:22:27.025300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe49e0c-22f2-4b89-b443-5f2b5e75ddc4 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:27.026336 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe49e0c-22f2-4b89-b443-5f2b5e75ddc4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:27.026814 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe49e0c-22f2-4b89-b443-5f2b5e75ddc4 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:27.028381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 31/62] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:27.033683 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1507b85-8ee2-4e8c-8a48-d1026a99e427 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:27.034560 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1507b85-8ee2-4e8c-8a48-d1026a99e427 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:27.035030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1507b85-8ee2-4e8c-8a48-d1026a99e427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:27.064210 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e1507b85-8ee2-4e8c-8a48-d1026a99e427 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:27.065746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e1507b85-8ee2-4e8c-8a48-d1026a99e427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:27.080557 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:27.159796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 32/63] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:27.194956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe49e0c-22f2-4b89-b443-5f2b5e75ddc4 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:27.304153 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3755747-aa63-486e-bb4a-0ff85bba7bc9 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:27.304616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3755747-aa63-486e-bb4a-0ff85bba7bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:27.304896 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3755747-aa63-486e-bb4a-0ff85bba7bc9 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:27.306414 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 33/64] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:27.311479 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d46aa2-69af-4376-9a0f-82b2ab18d562 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:27.311763 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d46aa2-69af-4376-9a0f-82b2ab18d562 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:27.312040 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d46aa2-69af-4376-9a0f-82b2ab18d562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:27.339239 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-88d46aa2-69af-4376-9a0f-82b2ab18d562 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:27.341673 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-88d46aa2-69af-4376-9a0f-82b2ab18d562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:27.359313 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe49e0c-22f2-4b89-b443-5f2b5e75ddc4 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:27.430820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e7bba-e690-48b4-b914-3b921cd59513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:27.431492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e7bba-e690-48b4-b914-3b921cd59513 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:27.431773 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e7bba-e690-48b4-b914-3b921cd59513 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:27.432047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e7bba-e690-48b4-b914-3b921cd59513 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:27.456009 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e7bba-e690-48b4-b914-3b921cd59513 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:27.461503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-cd7e7bba-e690-48b4-b914-3b921cd59513 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:22:27.461503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:27.461503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:27.461503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:27.461503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:27.461503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:27.461503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:27.461503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:27.461503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:27.461503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:27.461503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:27.461503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:27.461503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:27.461503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:27.461503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:27.461503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:27.462346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:27.462346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:27.462346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:27.462346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:27.462346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:27.462346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:27.462346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:27.462346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:27.462346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:27.462346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:27.462346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:27.462346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:27.462346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:27.462346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:27.462346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:27.462346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:27.463529 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:27.463529 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:27.463529 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:27.463529 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:27.463529 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:27.463529 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:27.463529 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:27.463529 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:27.463529 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:27.463529 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:27.463529 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:22:27.463529 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:22:27.463529 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 32/65] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:27.472197 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e7bba-e690-48b4-b914-3b921cd59513 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:27.481649 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 34/66] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:22:27.504728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d46aa2-69af-4376-9a0f-82b2ab18d562 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:27.523543 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:27.635662 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:27.652801 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e7bba-e690-48b4-b914-3b921cd59513 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:27.658428 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d46aa2-69af-4376-9a0f-82b2ab18d562 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:27.688790 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c6e73-82e3-486c-893c-cfc900ab96c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:27.689406 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c6e73-82e3-486c-893c-cfc900ab96c8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:27.689684 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c6e73-82e3-486c-893c-cfc900ab96c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:27.689952 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c6e73-82e3-486c-893c-cfc900ab96c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:27.690612 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c6e73-82e3-486c-893c-cfc900ab96c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:27.690911 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c6e73-82e3-486c-893c-cfc900ab96c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:27.692315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c6e73-82e3-486c-893c-cfc900ab96c8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:27.695715 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81942d5f-633f-4f8c-bbe5-4df81a15f549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:27.696355 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81942d5f-633f-4f8c-bbe5-4df81a15f549 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:27.696638 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81942d5f-633f-4f8c-bbe5-4df81a15f549 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:27.696914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81942d5f-633f-4f8c-bbe5-4df81a15f549 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:27.701602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 33/67] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 12:22:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:27.708468 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003bbe2-2e69-43eb-a2c2-33e5fb5568a7 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:27.708468 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003bbe2-2e69-43eb-a2c2-33e5fb5568a7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:27.708468 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003bbe2-2e69-43eb-a2c2-33e5fb5568a7 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:27.708468 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 34/68] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:27.708468 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75fadf-529a-454c-9916-8e00a92ccd26 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:27.708468 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75fadf-529a-454c-9916-8e00a92ccd26 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:27.708825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75fadf-529a-454c-9916-8e00a92ccd26 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:27.708825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 35/69] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:27.708825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d75fadf-529a-454c-9916-8e00a92ccd26 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:27.718287 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81942d5f-633f-4f8c-bbe5-4df81a15f549 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:27.723880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-81942d5f-633f-4f8c-bbe5-4df81a15f549 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:22:27.723880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:27.723880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:27.723880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:27.723880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:27.723880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:27.723880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:27.723880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:27.723880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:27.723880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:27.723880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:27.723880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:27.723880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:27.723880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:27.723880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:27.723880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:27.725574 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:27.725574 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:27.725574 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:27.725574 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:27.725574 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:27.725574 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:27.725574 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:27.725574 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:27.725574 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:27.725574 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:27.725574 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:27.725574 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:27.725574 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:27.725574 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:27.725574 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:27.725574 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:27.726521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:27.726521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:27.726521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:27.726521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:27.726521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:27.726521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:27.726521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:27.726521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:27.726521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:27.726521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:27.726521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:22:27.726521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:22:27.726521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 35/70] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:22:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:22:27.727446 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f179c-0484-4242-95e5-edf04ae3363f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:27.727741 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f179c-0484-4242-95e5-edf04ae3363f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:27.728022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f179c-0484-4242-95e5-edf04ae3363f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:27.745802 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-467f179c-0484-4242-95e5-edf04ae3363f None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:27.747014 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-467f179c-0484-4242-95e5-edf04ae3363f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:27.794373 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:27.813662 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d75fadf-529a-454c-9916-8e00a92ccd26 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:27.814279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 36/71] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:27.823346 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471d1c6-cc99-4bad-aa36-81f330d13d10 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:27.823346 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471d1c6-cc99-4bad-aa36-81f330d13d10 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:27.823346 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471d1c6-cc99-4bad-aa36-81f330d13d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:27.835039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d471d1c6-cc99-4bad-aa36-81f330d13d10 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:27.836303 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d471d1c6-cc99-4bad-aa36-81f330d13d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:27.847358 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428834d-4068-487f-9c84-17467073cf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:27.847991 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428834d-4068-487f-9c84-17467073cf95 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:27.848271 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428834d-4068-487f-9c84-17467073cf95 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:27.848547 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428834d-4068-487f-9c84-17467073cf95 None admin] PATH_INFO: `/v3/users/a43f0c8b634b4f2997cbdf44c7dd2c5a` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:27.863605 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428834d-4068-487f-9c84-17467073cf95 None admin] RBAC: Authorizing `identity:get_user(user_id=a43f0c8b634b4f2997cbdf44c7dd2c5a)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:27.865048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428834d-4068-487f-9c84-17467073cf95 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:27.866891 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 36/72] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:27 2022] GET /identity/v3/users/a43f0c8b634b4f2997cbdf44c7dd2c5a => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:27.868051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc1341-b998-4067-98ee-515a43a3df7d None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:27.868335 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc1341-b998-4067-98ee-515a43a3df7d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:27.868610 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc1341-b998-4067-98ee-515a43a3df7d None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:27.870382 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 37/73] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:27.871084 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc1341-b998-4067-98ee-515a43a3df7d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:27.901893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:27.906748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 37/74] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:27.927246 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471d1c6-cc99-4bad-aa36-81f330d13d10 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:27.972818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc1341-b998-4067-98ee-515a43a3df7d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:28.006174 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471d1c6-cc99-4bad-aa36-81f330d13d10 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:28.007732 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9ecbe-481f-44fc-b84e-ea71e8544219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:28.008345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9ecbe-481f-44fc-b84e-ea71e8544219 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:28.008625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9ecbe-481f-44fc-b84e-ea71e8544219 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:28.008898 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9ecbe-481f-44fc-b84e-ea71e8544219 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:28.009558 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9ecbe-481f-44fc-b84e-ea71e8544219 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:28.009869 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9ecbe-481f-44fc-b84e-ea71e8544219 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:28.011268 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9ecbe-481f-44fc-b84e-ea71e8544219 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:28.019202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 38/75] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:28.020138 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9ecbe-481f-44fc-b84e-ea71e8544219 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:28.022689 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:28.040315 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756186a-b7de-4aae-adaa-82f8c91aab0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:28.040955 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756186a-b7de-4aae-adaa-82f8c91aab0d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:28.041247 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756186a-b7de-4aae-adaa-82f8c91aab0d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:28.041525 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756186a-b7de-4aae-adaa-82f8c91aab0d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:28.051820 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1756186a-b7de-4aae-adaa-82f8c91aab0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:28.057343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-1756186a-b7de-4aae-adaa-82f8c91aab0d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:22:28.057343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:28.057343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:28.057343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:28.057343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:28.057343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:28.057343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:28.057343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:28.057343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:28.057343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:28.057343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:28.057343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:28.057343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:28.057343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:28.057343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:28.057343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:28.058190 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:28.058190 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:28.058190 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:28.058190 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:28.058190 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:28.058190 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:28.058190 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:28.058190 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:28.058190 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:28.058190 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:28.058190 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:28.058190 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:28.058190 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:28.058190 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:28.058190 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:28.058190 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:28.064206 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:28.064206 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:28.064206 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:28.064206 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:28.064206 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:28.064206 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:28.064206 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:28.064206 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:28.064206 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:28.064206 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:28.064206 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:22:28.064206 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:22:28.064206 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 38/76] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 12:22:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:28.064206 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15a785-5b7e-43ef-acce-8c8c51d38c62 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:28.064879 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15a785-5b7e-43ef-acce-8c8c51d38c62 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:28.064879 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15a785-5b7e-43ef-acce-8c8c51d38c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:28.077534 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ad15a785-5b7e-43ef-acce-8c8c51d38c62 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:28.078732 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ad15a785-5b7e-43ef-acce-8c8c51d38c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:28.121696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9ecbe-481f-44fc-b84e-ea71e8544219 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:28.132126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:28.145322 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 39/77] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:28.146134 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15a785-5b7e-43ef-acce-8c8c51d38c62 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:28.157310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed5aa8-1ec8-4ab6-8fe5-a66a8aed74b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:28.157867 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed5aa8-1ec8-4ab6-8fe5-a66a8aed74b5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:28.158214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed5aa8-1ec8-4ab6-8fe5-a66a8aed74b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:28.158496 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed5aa8-1ec8-4ab6-8fe5-a66a8aed74b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:28.227583 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed5aa8-1ec8-4ab6-8fe5-a66a8aed74b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:28.229054 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed5aa8-1ec8-4ab6-8fe5-a66a8aed74b5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:28.230869 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:28.246523 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15a785-5b7e-43ef-acce-8c8c51d38c62 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:28.280978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc577bb-1d3a-45ff-b946-d02929757b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:28.281617 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc577bb-1d3a-45ff-b946-d02929757b60 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:28.281897 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc577bb-1d3a-45ff-b946-d02929757b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:28.282171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc577bb-1d3a-45ff-b946-d02929757b60 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:28.293064 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc577bb-1d3a-45ff-b946-d02929757b60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:28.298132 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-9fc577bb-1d3a-45ff-b946-d02929757b60 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:22:28.298132 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:28.298132 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:28.298132 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:28.298132 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:28.298132 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:28.298132 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:28.298132 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:28.298132 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:28.298132 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:28.298132 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:28.298132 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:28.298132 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:28.298132 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:28.298132 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:28.298132 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:28.299042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:28.299042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:28.299042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:28.299042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:28.299042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:28.299042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:28.299042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:28.299042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:28.299042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:28.299042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:28.299042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:28.299042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:28.299042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:28.299042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:28.299042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:28.299042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:28.299920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:28.299920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:28.299920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:28.299920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:28.299920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:28.299920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:28.299920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:28.299920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:28.299920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:28.299920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:28.299920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:22:28.299920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:22:28.299920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 40/78] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:28.301875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc577bb-1d3a-45ff-b946-d02929757b60 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:28.330478 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 39/79] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:22:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:22:28.331347 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed5aa8-1ec8-4ab6-8fe5-a66a8aed74b5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:28.349593 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:28.401620 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc577bb-1d3a-45ff-b946-d02929757b60 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:28.430581 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed5aa8-1ec8-4ab6-8fe5-a66a8aed74b5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:28.432061 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:28.434143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028512b9-a85c-4c76-bfdb-820e652e823f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:28.434775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028512b9-a85c-4c76-bfdb-820e652e823f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:28.435058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028512b9-a85c-4c76-bfdb-820e652e823f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:28.435373 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028512b9-a85c-4c76-bfdb-820e652e823f None admin] PATH_INFO: `/v3/users/903d09eed7334b43af767deea1a6bd32` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:28.436503 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028512b9-a85c-4c76-bfdb-820e652e823f None admin] RBAC: Authorizing `identity:get_user(user_id=903d09eed7334b43af767deea1a6bd32)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:28.437976 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028512b9-a85c-4c76-bfdb-820e652e823f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:28.439790 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 41/80] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:28 2022] GET /identity/v3/users/903d09eed7334b43af767deea1a6bd32 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:28.440427 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028512b9-a85c-4c76-bfdb-820e652e823f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:28.469318 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7273bdb2-a589-40ea-9bb7-8de8e52c7ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:28.469933 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273bdb2-a589-40ea-9bb7-8de8e52c7ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:28.470219 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273bdb2-a589-40ea-9bb7-8de8e52c7ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:28.470727 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273bdb2-a589-40ea-9bb7-8de8e52c7ea9 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:28.471152 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7273bdb2-a589-40ea-9bb7-8de8e52c7ea9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:28.471489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7273bdb2-a589-40ea-9bb7-8de8e52c7ea9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:28.472863 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7273bdb2-a589-40ea-9bb7-8de8e52c7ea9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:28.481248 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 40/81] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:22:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:28.482035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7273bdb2-a589-40ea-9bb7-8de8e52c7ea9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:28.542875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028512b9-a85c-4c76-bfdb-820e652e823f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:28.559011 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:28.576497 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7664ed55-debc-425d-a568-8157ca978df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:28.577123 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664ed55-debc-425d-a568-8157ca978df6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:28.577402 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664ed55-debc-425d-a568-8157ca978df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:28.577672 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664ed55-debc-425d-a568-8157ca978df6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:28.578765 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7664ed55-debc-425d-a568-8157ca978df6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:28.580193 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7273bdb2-a589-40ea-9bb7-8de8e52c7ea9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:28.580310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7664ed55-debc-425d-a568-8157ca978df6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:28.581882 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 42/82] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:22:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:28.583989 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7664ed55-debc-425d-a568-8157ca978df6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:28.614381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0edca32-a5c7-4824-86fb-b3ccd9e5ccaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:28.614993 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edca32-a5c7-4824-86fb-b3ccd9e5ccaa None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:28.615329 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edca32-a5c7-4824-86fb-b3ccd9e5ccaa None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:28.615655 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edca32-a5c7-4824-86fb-b3ccd9e5ccaa None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:28.616395 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0edca32-a5c7-4824-86fb-b3ccd9e5ccaa None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:28.616706 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0edca32-a5c7-4824-86fb-b3ccd9e5ccaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:28.618090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0edca32-a5c7-4824-86fb-b3ccd9e5ccaa None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:28.626151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 41/83] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 12:22:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:28.626931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0edca32-a5c7-4824-86fb-b3ccd9e5ccaa None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:28.654423 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:28.688519 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7664ed55-debc-425d-a568-8157ca978df6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:28.721344 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5092b9-94c4-4443-99df-1c70c4ed7ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:28.721980 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5092b9-94c4-4443-99df-1c70c4ed7ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:28.722260 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5092b9-94c4-4443-99df-1c70c4ed7ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:28.722532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5092b9-94c4-4443-99df-1c70c4ed7ba4 None admin] PATH_INFO: `/v3/projects/9c9a0023662b41afa1d7bec05fd12eda` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:28.723691 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5092b9-94c4-4443-99df-1c70c4ed7ba4 None admin] RBAC: Authorizing `identity:get_project(project_id=9c9a0023662b41afa1d7bec05fd12eda)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:28.725707 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5092b9-94c4-4443-99df-1c70c4ed7ba4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:28.726290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0edca32-a5c7-4824-86fb-b3ccd9e5ccaa None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:28.727378 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 43/84] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:22:28 2022] GET /identity/v3/projects/9c9a0023662b41afa1d7bec05fd12eda => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:28.728138 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5092b9-94c4-4443-99df-1c70c4ed7ba4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:28.765793 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21b27f-346a-4a34-aa04-05cd4a9a98ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:28.766409 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21b27f-346a-4a34-aa04-05cd4a9a98ae None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:28.766694 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21b27f-346a-4a34-aa04-05cd4a9a98ae None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:28.766977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21b27f-346a-4a34-aa04-05cd4a9a98ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:28.780408 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:28.827019 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5092b9-94c4-4443-99df-1c70c4ed7ba4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:28.835358 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21b27f-346a-4a34-aa04-05cd4a9a98ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:28.836837 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21b27f-346a-4a34-aa04-05cd4a9a98ae None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:28.861140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a4d2b-7919-4362-ac6d-b87c970a23ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:28.861804 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a4d2b-7919-4362-ac6d-b87c970a23ad None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:28.862180 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a4d2b-7919-4362-ac6d-b87c970a23ad None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:28.862464 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a4d2b-7919-4362-ac6d-b87c970a23ad None admin] PATH_INFO: `/v3/users/a43f0c8b634b4f2997cbdf44c7dd2c5a` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:28.862567 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:28.863629 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a4d2b-7919-4362-ac6d-b87c970a23ad None admin] RBAC: Authorizing `identity:get_user(user_id=a43f0c8b634b4f2997cbdf44c7dd2c5a)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:28.865087 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a4d2b-7919-4362-ac6d-b87c970a23ad None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:28.867010 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 44/85] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:28 2022] GET /identity/v3/users/a43f0c8b634b4f2997cbdf44c7dd2c5a => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:28.871962 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a4d2b-7919-4362-ac6d-b87c970a23ad None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:28.939034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 42/86] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:22:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:22:28.939842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21b27f-346a-4a34-aa04-05cd4a9a98ae None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:28.968563 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a4d2b-7919-4362-ac6d-b87c970a23ad None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:28.988175 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:29.002868 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e6645-e141-41d9-b964-cee966cfac15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:29.003508 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e6645-e141-41d9-b964-cee966cfac15 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:29.003791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e6645-e141-41d9-b964-cee966cfac15 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:29.004065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e6645-e141-41d9-b964-cee966cfac15 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:29.004706 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78e6645-e141-41d9-b964-cee966cfac15 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:29.006728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78e6645-e141-41d9-b964-cee966cfac15 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:29.038538 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21b27f-346a-4a34-aa04-05cd4a9a98ae None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:29.063157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:29.072275 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee27c4-881b-4bf5-8244-1073f993eaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:29.072939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee27c4-881b-4bf5-8244-1073f993eaa5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:29.073231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee27c4-881b-4bf5-8244-1073f993eaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:29.073507 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee27c4-881b-4bf5-8244-1073f993eaa5 None admin] PATH_INFO: `/v3/users/a43f0c8b634b4f2997cbdf44c7dd2c5a` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:29.074683 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee27c4-881b-4bf5-8244-1073f993eaa5 None admin] RBAC: Authorizing `identity:get_user(user_id=a43f0c8b634b4f2997cbdf44c7dd2c5a)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:29.076195 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee27c4-881b-4bf5-8244-1073f993eaa5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:29.077771 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 43/87] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:28 2022] GET /identity/v3/users/a43f0c8b634b4f2997cbdf44c7dd2c5a => generated 275 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:29.078533 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee27c4-881b-4bf5-8244-1073f993eaa5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:29.111205 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 45/88] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 12:22:28 2022] POST /identity/v3/users => generated 289 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:22:29.111205 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e6645-e141-41d9-b964-cee966cfac15 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:29.193459 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee27c4-881b-4bf5-8244-1073f993eaa5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:29.215819 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e6645-e141-41d9-b964-cee966cfac15 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:29.233805 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78769c3c-d8dc-492a-8d17-af6f494741c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:29.234685 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78769c3c-d8dc-492a-8d17-af6f494741c1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:29.235110 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78769c3c-d8dc-492a-8d17-af6f494741c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:29.235581 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78769c3c-d8dc-492a-8d17-af6f494741c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:29.236468 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78769c3c-d8dc-492a-8d17-af6f494741c1 None admin] RBAC: Adding query filter params (role.id=872ba0c4212a4684b539a68fc2c20b25, scope.project.id=9c9a0023662b41afa1d7bec05fd12eda, user.id=a43f0c8b634b4f2997cbdf44c7dd2c5a) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:29.236927 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78769c3c-d8dc-492a-8d17-af6f494741c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:29.238528 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78769c3c-d8dc-492a-8d17-af6f494741c1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:29.255371 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 44/89] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:22:29 2022] GET /identity/v3/role_assignments?user.id=a43f0c8b634b4f2997cbdf44c7dd2c5a&scope.project.id=9c9a0023662b41afa1d7bec05fd12eda&role.id=872ba0c4212a4684b539a68fc2c20b25 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:29.257188 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78769c3c-d8dc-492a-8d17-af6f494741c1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:29.261183 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c2fbe-9406-4a73-8cc0-1adccafb6c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:29.261826 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c2fbe-9406-4a73-8cc0-1adccafb6c39 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:29.262119 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c2fbe-9406-4a73-8cc0-1adccafb6c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:29.262392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c2fbe-9406-4a73-8cc0-1adccafb6c39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:29.327648 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:29.355493 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:29.363760 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c2fbe-9406-4a73-8cc0-1adccafb6c39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:29.365407 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c2fbe-9406-4a73-8cc0-1adccafb6c39 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:29.419676 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78769c3c-d8dc-492a-8d17-af6f494741c1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:29.468358 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c599e-8895-4315-80b7-37b2c950489a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:29.469116 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c599e-8895-4315-80b7-37b2c950489a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:29.469407 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c599e-8895-4315-80b7-37b2c950489a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:29.469808 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c599e-8895-4315-80b7-37b2c950489a None admin] PATH_INFO: `/v3/projects/9c9a0023662b41afa1d7bec05fd12eda` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:29.470990 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9c599e-8895-4315-80b7-37b2c950489a None admin] RBAC: Authorizing `identity:get_project(project_id=9c9a0023662b41afa1d7bec05fd12eda)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:29.472558 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9c599e-8895-4315-80b7-37b2c950489a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:29.474440 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 45/90] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:22:29 2022] GET /identity/v3/projects/9c9a0023662b41afa1d7bec05fd12eda => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:29.475045 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c599e-8895-4315-80b7-37b2c950489a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:29.500729 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 46/91] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:22:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:22:29.501594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c2fbe-9406-4a73-8cc0-1adccafb6c39 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:29.586277 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c599e-8895-4315-80b7-37b2c950489a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:29.586516 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:29.612157 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c2fbe-9406-4a73-8cc0-1adccafb6c39 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:29.613706 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:29.624447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e69548-0ba0-45f5-b0cd-caebf1a8fec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:29.628213 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e69548-0ba0-45f5-b0cd-caebf1a8fec1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:29.628213 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e69548-0ba0-45f5-b0cd-caebf1a8fec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:29.628213 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e69548-0ba0-45f5-b0cd-caebf1a8fec1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:29.655343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc28391-4c9f-4c61-bc9a-3461b650ec4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:29.656254 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc28391-4c9f-4c61-bc9a-3461b650ec4c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:29.656736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc28391-4c9f-4c61-bc9a-3461b650ec4c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:29.657281 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc28391-4c9f-4c61-bc9a-3461b650ec4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:29.658292 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc28391-4c9f-4c61-bc9a-3461b650ec4c None admin] RBAC: Adding query filter params (role.id=bffbe227cda1418cbb58394ddff6f861, scope.project.id=9c9a0023662b41afa1d7bec05fd12eda, user.id=903d09eed7334b43af767deea1a6bd32) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:29.658780 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc28391-4c9f-4c61-bc9a-3461b650ec4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:29.660562 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc28391-4c9f-4c61-bc9a-3461b650ec4c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:29.668523 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 47/92] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:22:29 2022] GET /identity/v3/role_assignments?user.id=903d09eed7334b43af767deea1a6bd32&scope.project.id=9c9a0023662b41afa1d7bec05fd12eda&role.id=bffbe227cda1418cbb58394ddff6f861 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:29.670365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc28391-4c9f-4c61-bc9a-3461b650ec4c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:29.705845 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e69548-0ba0-45f5-b0cd-caebf1a8fec1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:29.707521 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e69548-0ba0-45f5-b0cd-caebf1a8fec1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:29.788731 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc28391-4c9f-4c61-bc9a-3461b650ec4c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:29.819969 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 46/93] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:22:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:22:29.825712 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e69548-0ba0-45f5-b0cd-caebf1a8fec1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:29.829404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea85531d-f2bb-4c3b-8862-0b4ebd934698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:29.830290 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85531d-f2bb-4c3b-8862-0b4ebd934698 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:29.830750 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85531d-f2bb-4c3b-8862-0b4ebd934698 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:29.831257 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85531d-f2bb-4c3b-8862-0b4ebd934698 None admin] PATH_INFO: `/v3/projects/9c9a0023662b41afa1d7bec05fd12eda` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:29.832681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea85531d-f2bb-4c3b-8862-0b4ebd934698 None admin] RBAC: Authorizing `identity:get_project(project_id=9c9a0023662b41afa1d7bec05fd12eda)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:29.834526 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea85531d-f2bb-4c3b-8862-0b4ebd934698 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:29.836664 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 48/94] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:22:29 2022] GET /identity/v3/projects/9c9a0023662b41afa1d7bec05fd12eda => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:29.844279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea85531d-f2bb-4c3b-8862-0b4ebd934698 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:29.859729 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:29.863511 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:29.965719 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e69548-0ba0-45f5-b0cd-caebf1a8fec1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:29.968357 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea85531d-f2bb-4c3b-8862-0b4ebd934698 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:30.008916 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb1fa9-0be1-46d0-9ba6-88e865d6dc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:30.010440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb1fa9-0be1-46d0-9ba6-88e865d6dc55 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:30.011140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb1fa9-0be1-46d0-9ba6-88e865d6dc55 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:30.011648 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb1fa9-0be1-46d0-9ba6-88e865d6dc55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:30.012579 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbb1fa9-0be1-46d0-9ba6-88e865d6dc55 None admin] RBAC: Adding query filter params (role.id=0f58d2782a2840648bbbc2ab26ec02df, scope.project.id=9c9a0023662b41afa1d7bec05fd12eda, user.id=a43f0c8b634b4f2997cbdf44c7dd2c5a) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:30.013204 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbb1fa9-0be1-46d0-9ba6-88e865d6dc55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:30.014962 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbb1fa9-0be1-46d0-9ba6-88e865d6dc55 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:30.021669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 49/95] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:22:29 2022] GET /identity/v3/role_assignments?user.id=a43f0c8b634b4f2997cbdf44c7dd2c5a&scope.project.id=9c9a0023662b41afa1d7bec05fd12eda&role.id=0f58d2782a2840648bbbc2ab26ec02df => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:30.029933 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cda49f-a186-4da5-b48c-9ad858c97a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:30.030611 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cda49f-a186-4da5-b48c-9ad858c97a5d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:30.030895 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cda49f-a186-4da5-b48c-9ad858c97a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:30.031201 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cda49f-a186-4da5-b48c-9ad858c97a5d None admin] PATH_INFO: `/v3/projects/f9ec7e64955644058c55670463303dd3` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:30.036178 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cda49f-a186-4da5-b48c-9ad858c97a5d None admin] RBAC: Authorizing `identity:get_project(project_id=f9ec7e64955644058c55670463303dd3)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:30.038073 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cda49f-a186-4da5-b48c-9ad858c97a5d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:30.040247 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 47/96] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:22:29 2022] GET /identity/v3/projects/f9ec7e64955644058c55670463303dd3 => generated 312 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:30.046610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb1fa9-0be1-46d0-9ba6-88e865d6dc55 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:30.127544 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:30.164031 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb1fa9-0be1-46d0-9ba6-88e865d6dc55 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:30.202438 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffecbba-82f3-4671-bb11-8d45f55b4834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:30.203439 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffecbba-82f3-4671-bb11-8d45f55b4834 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:30.203924 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffecbba-82f3-4671-bb11-8d45f55b4834 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:30.204429 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffecbba-82f3-4671-bb11-8d45f55b4834 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:30.205388 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffecbba-82f3-4671-bb11-8d45f55b4834 None admin] RBAC: Adding query filter params (role.id=872ba0c4212a4684b539a68fc2c20b25, scope.project.id=f9ec7e64955644058c55670463303dd3, user.id=a43f0c8b634b4f2997cbdf44c7dd2c5a) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:30.205905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffecbba-82f3-4671-bb11-8d45f55b4834 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:30.207629 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffecbba-82f3-4671-bb11-8d45f55b4834 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:30.215156 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 50/97] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:22:30 2022] GET /identity/v3/role_assignments?user.id=a43f0c8b634b4f2997cbdf44c7dd2c5a&scope.project.id=f9ec7e64955644058c55670463303dd3&role.id=872ba0c4212a4684b539a68fc2c20b25 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:31.945255 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b07e4-6b79-4e68-b8c5-04c83a6402ef None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:31.952326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b07e4-6b79-4e68-b8c5-04c83a6402ef None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:31.952326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b07e4-6b79-4e68-b8c5-04c83a6402ef None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:31.952326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 48/98] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:31.952876 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f994f-153a-4cd8-b428-6c2564518767 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:31.953687 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f994f-153a-4cd8-b428-6c2564518767 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:31.954221 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f994f-153a-4cd8-b428-6c2564518767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:31.977587 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-312f994f-153a-4cd8-b428-6c2564518767 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:31.979104 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-312f994f-153a-4cd8-b428-6c2564518767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:32.113041 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 51/99] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:32.148904 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b07e4-6b79-4e68-b8c5-04c83a6402ef None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:32.270116 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193854d-9d9f-42cf-9cab-7da97320d0e2 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:32.270430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193854d-9d9f-42cf-9cab-7da97320d0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:32.270696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193854d-9d9f-42cf-9cab-7da97320d0e2 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:32.272804 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 52/100] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:32.279365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d479afa-9f15-47f2-be7b-c2f9e00d83e9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:32.279463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d479afa-9f15-47f2-be7b-c2f9e00d83e9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:32.283209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d479afa-9f15-47f2-be7b-c2f9e00d83e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:32.283332 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:32.304993 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b07e4-6b79-4e68-b8c5-04c83a6402ef None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:32.320256 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3d479afa-9f15-47f2-be7b-c2f9e00d83e9 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:32.321905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3d479afa-9f15-47f2-be7b-c2f9e00d83e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:32.382117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b69a3ce-e6db-4fe7-9e51-1af775d97521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:32.382779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69a3ce-e6db-4fe7-9e51-1af775d97521 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:32.383059 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69a3ce-e6db-4fe7-9e51-1af775d97521 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:32.383359 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69a3ce-e6db-4fe7-9e51-1af775d97521 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:32.404127 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b69a3ce-e6db-4fe7-9e51-1af775d97521 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:32.409143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-5b69a3ce-e6db-4fe7-9e51-1af775d97521 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:22:32.409143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:32.409143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:32.409143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:32.409143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:32.409143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:32.409143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:32.409143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:32.409143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:32.409143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:32.409143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:32.409143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:32.409143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:32.409143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:32.409143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:32.409143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:32.410012 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:32.410012 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:32.410012 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:32.410012 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:32.410012 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:32.410012 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:32.410012 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:32.410012 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:32.410012 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:32.410012 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:32.410012 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:32.410012 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:32.410012 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:32.410012 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:32.410012 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:32.410012 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:32.411922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:32.411922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:32.411922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:32.411922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:32.411922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:32.411922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:32.411922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:32.411922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:32.411922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:32.411922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:32.411922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:22:32.411922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:22:32.411922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 49/101] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:32.416917 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b69a3ce-e6db-4fe7-9e51-1af775d97521 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:32.470820 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:32.485465 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 53/102] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:32.529789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d479afa-9f15-47f2-be7b-c2f9e00d83e9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:32.605709 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b69a3ce-e6db-4fe7-9e51-1af775d97521 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:32.667495 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78adb0a7-f4c3-4f67-9bb3-53f801730b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:32.668115 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adb0a7-f4c3-4f67-9bb3-53f801730b62 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:32.668408 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adb0a7-f4c3-4f67-9bb3-53f801730b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:32.669503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adb0a7-f4c3-4f67-9bb3-53f801730b62 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:32.670136 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78adb0a7-f4c3-4f67-9bb3-53f801730b62 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:32.670445 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78adb0a7-f4c3-4f67-9bb3-53f801730b62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:32.671866 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78adb0a7-f4c3-4f67-9bb3-53f801730b62 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:32.681238 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 50/103] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 12:22:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:32.682326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95500881-080e-4dda-8c72-06750f32fe72 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:32.682622 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95500881-080e-4dda-8c72-06750f32fe72 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:32.682993 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95500881-080e-4dda-8c72-06750f32fe72 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:32.684312 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 51/104] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:32.685164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765e4ba-cc6b-479c-8ad5-8b1434e6aecb None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:32.685448 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765e4ba-cc6b-479c-8ad5-8b1434e6aecb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:32.685721 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765e4ba-cc6b-479c-8ad5-8b1434e6aecb None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:32.686904 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 52/105] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:32.691571 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2765e4ba-cc6b-479c-8ad5-8b1434e6aecb None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:32.711775 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:32.728662 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d479afa-9f15-47f2-be7b-c2f9e00d83e9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:32.774809 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b3860-4dd5-4c07-934f-06aab26c1e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:32.775880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b3860-4dd5-4c07-934f-06aab26c1e61 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:32.776366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b3860-4dd5-4c07-934f-06aab26c1e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:32.777256 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b3860-4dd5-4c07-934f-06aab26c1e61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:32.778606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b3860-4dd5-4c07-934f-06aab26c1e61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:32.780724 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b3860-4dd5-4c07-934f-06aab26c1e61 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:32.782460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 54/106] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:22:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:32.784753 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce159d-d33a-4e5a-9557-b617bb164720 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:32.784961 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce159d-d33a-4e5a-9557-b617bb164720 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:32.785944 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce159d-d33a-4e5a-9557-b617bb164720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:32.805091 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-cbce159d-d33a-4e5a-9557-b617bb164720 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:32.806375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cbce159d-d33a-4e5a-9557-b617bb164720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:32.815309 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2765e4ba-cc6b-479c-8ad5-8b1434e6aecb None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:32.863004 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c553b5f6-c346-4857-a93d-ace185893151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:32.864522 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c553b5f6-c346-4857-a93d-ace185893151 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:32.865312 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c553b5f6-c346-4857-a93d-ace185893151 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:32.866426 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c553b5f6-c346-4857-a93d-ace185893151 None admin] PATH_INFO: `/v3/users/9bdcd0c1fbd34f56abe6a3fd9c128c69` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:32.870735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:32.880042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 55/107] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:32.887012 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c553b5f6-c346-4857-a93d-ace185893151 None admin] RBAC: Authorizing `identity:get_user(user_id=9bdcd0c1fbd34f56abe6a3fd9c128c69)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:32.887681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce8506-55ca-4bde-962c-79e0712a4073 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:32.887986 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce8506-55ca-4bde-962c-79e0712a4073 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:32.888267 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce8506-55ca-4bde-962c-79e0712a4073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:32.889436 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c553b5f6-c346-4857-a93d-ace185893151 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:32.891463 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 53/108] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:32 2022] GET /identity/v3/users/9bdcd0c1fbd34f56abe6a3fd9c128c69 => generated 289 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:32.892550 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c553b5f6-c346-4857-a93d-ace185893151 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:32.918981 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-40ce8506-55ca-4bde-962c-79e0712a4073 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:32.920241 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-40ce8506-55ca-4bde-962c-79e0712a4073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:33.003248 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 56/109] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:33.003248 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce8506-55ca-4bde-962c-79e0712a4073 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:33.022036 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:33.054247 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c553b5f6-c346-4857-a93d-ace185893151 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:33.094861 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57731407-ecc1-4b84-bd52-ce5f53d6aabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:33.095880 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57731407-ecc1-4b84-bd52-ce5f53d6aabf None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:33.096180 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57731407-ecc1-4b84-bd52-ce5f53d6aabf None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:33.096461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57731407-ecc1-4b84-bd52-ce5f53d6aabf None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:33.097428 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57731407-ecc1-4b84-bd52-ce5f53d6aabf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:33.099182 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57731407-ecc1-4b84-bd52-ce5f53d6aabf None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:33.119405 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce8506-55ca-4bde-962c-79e0712a4073 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:33.143709 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:33.154134 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7601f4b7-abfe-41e3-95a2-b947364e6b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:33.154752 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601f4b7-abfe-41e3-95a2-b947364e6b5c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:33.155037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601f4b7-abfe-41e3-95a2-b947364e6b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:33.155350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601f4b7-abfe-41e3-95a2-b947364e6b5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:33.190909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 54/110] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 10 12:22:32 2022] POST /identity/v3/projects => generated 302 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:22:33.192196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57731407-ecc1-4b84-bd52-ce5f53d6aabf None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:33.234478 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7601f4b7-abfe-41e3-95a2-b947364e6b5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:33.236174 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7601f4b7-abfe-41e3-95a2-b947364e6b5c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:33.331325 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57731407-ecc1-4b84-bd52-ce5f53d6aabf None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:33.368612 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:33.370817 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 57/111] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:22:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:22:33.371726 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7601f4b7-abfe-41e3-95a2-b947364e6b5c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:33.382066 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3fa949-5b6d-48ae-a30d-ec8744864516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:33.382066 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fa949-5b6d-48ae-a30d-ec8744864516 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:33.382066 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fa949-5b6d-48ae-a30d-ec8744864516 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:33.382066 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fa949-5b6d-48ae-a30d-ec8744864516 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:33.394374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3fa949-5b6d-48ae-a30d-ec8744864516 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:33.400624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-9b3fa949-5b6d-48ae-a30d-ec8744864516 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:22:33.400624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:33.400624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:33.400624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:33.400624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:33.400624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:33.400624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:33.400624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:33.400624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:33.400624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:33.400624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:33.400624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:33.400624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:33.400624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:33.400624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:33.400624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:33.401483 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:33.401483 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:33.401483 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:33.401483 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:33.401483 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:33.401483 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:33.401483 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:33.401483 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:33.401483 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:33.401483 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:33.401483 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:33.401483 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:33.401483 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:33.401483 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:33.401483 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:33.401483 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:33.402237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:33.402237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:33.402237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:33.402237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:33.402237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:33.402237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:33.402237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:33.402237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:33.402237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:33.402237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:33.402237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:22:33.402237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:22:33.404716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:33.404905 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 55/112] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:33.406505 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad6d7c-1e67-47db-b410-af37dd3989bd None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:33.406798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad6d7c-1e67-47db-b410-af37dd3989bd None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:33.407326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad6d7c-1e67-47db-b410-af37dd3989bd None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:33.409584 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 56/113] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:33.410577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad6d7c-1e67-47db-b410-af37dd3989bd None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:33.491008 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7601f4b7-abfe-41e3-95a2-b947364e6b5c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:33.521253 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad6d7c-1e67-47db-b410-af37dd3989bd None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:33.526242 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb98dd2-ee1f-4241-973d-f81628aa4bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:33.526883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb98dd2-ee1f-4241-973d-f81628aa4bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:33.527168 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb98dd2-ee1f-4241-973d-f81628aa4bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:33.527553 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb98dd2-ee1f-4241-973d-f81628aa4bb9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:33.537950 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb98dd2-ee1f-4241-973d-f81628aa4bb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:33.542633 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-afb98dd2-ee1f-4241-973d-f81628aa4bb9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:22:33.542633 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:33.542633 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:33.542633 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:33.542633 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:33.542633 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:33.542633 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:33.542633 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:33.542633 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:33.542633 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:33.542633 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:33.542633 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:33.542633 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:33.542633 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:33.542633 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:33.542633 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:33.543565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:33.543565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:33.543565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:33.543565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:33.543565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:33.543565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:33.543565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:33.543565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:33.543565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:33.543565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:33.543565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:33.543565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:33.543565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:33.543565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:33.543565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:33.543565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:33.544358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:33.544358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:33.544358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:33.544358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:33.544358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:33.544358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:33.544358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:33.544358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:33.544358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:33.544358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:33.544358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:22:33.544358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:22:33.544358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 58/114] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:22:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:33.544969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546492ee-2b2c-4648-a5c8-f6627f145b94 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:33.545267 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546492ee-2b2c-4648-a5c8-f6627f145b94 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:33.545565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546492ee-2b2c-4648-a5c8-f6627f145b94 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:33.551263 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 59/115] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:33.551263 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546492ee-2b2c-4648-a5c8-f6627f145b94 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:33.565529 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb47c1-729c-43f2-9fc9-c65dafce1816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:33.566240 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb47c1-729c-43f2-9fc9-c65dafce1816 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:33.566502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb47c1-729c-43f2-9fc9-c65dafce1816 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:33.566786 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb47c1-729c-43f2-9fc9-c65dafce1816 None admin] PATH_INFO: `/v3/projects/9c9a0023662b41afa1d7bec05fd12eda` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:33.567985 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb47c1-729c-43f2-9fc9-c65dafce1816 None admin] RBAC: Authorizing `identity:get_project(project_id=9c9a0023662b41afa1d7bec05fd12eda)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:33.569491 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb47c1-729c-43f2-9fc9-c65dafce1816 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:33.571223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 57/116] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:22:33 2022] GET /identity/v3/projects/9c9a0023662b41afa1d7bec05fd12eda => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:33.572344 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099d1cf-14ed-4271-a15f-62b81997974d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:33.572636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099d1cf-14ed-4271-a15f-62b81997974d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:33.572913 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099d1cf-14ed-4271-a15f-62b81997974d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:33.605379 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d099d1cf-14ed-4271-a15f-62b81997974d None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:33.606786 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d099d1cf-14ed-4271-a15f-62b81997974d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:33.644452 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:33.659899 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:33.663477 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546492ee-2b2c-4648-a5c8-f6627f145b94 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:33.678634 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 58/117] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:33.680200 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099d1cf-14ed-4271-a15f-62b81997974d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:33.712279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f3c65-b7e5-415d-ab75-21dadbb7f608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:33.712898 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f3c65-b7e5-415d-ab75-21dadbb7f608 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:33.713179 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f3c65-b7e5-415d-ab75-21dadbb7f608 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:33.713449 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f3c65-b7e5-415d-ab75-21dadbb7f608 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:33.714107 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f3c65-b7e5-415d-ab75-21dadbb7f608 None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:33.714410 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f3c65-b7e5-415d-ab75-21dadbb7f608 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:33.715827 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f3c65-b7e5-415d-ab75-21dadbb7f608 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:33.726017 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 60/118] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 12:22:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:33.727260 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5d0ed-627c-4bb5-9324-947cea91e243 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:33.727553 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5d0ed-627c-4bb5-9324-947cea91e243 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:33.727832 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5d0ed-627c-4bb5-9324-947cea91e243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:33.748185 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-81a5d0ed-627c-4bb5-9324-947cea91e243 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:33.749362 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-81a5d0ed-627c-4bb5-9324-947cea91e243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:33.795900 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099d1cf-14ed-4271-a15f-62b81997974d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:33.823772 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 61/119] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:33.824426 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5d0ed-627c-4bb5-9324-947cea91e243 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:33.844687 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4081fd8-45fb-425a-9d49-287d18032e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:33.845315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4081fd8-45fb-425a-9d49-287d18032e8f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:33.845647 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4081fd8-45fb-425a-9d49-287d18032e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:33.845931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4081fd8-45fb-425a-9d49-287d18032e8f None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:33.847045 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4081fd8-45fb-425a-9d49-287d18032e8f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:33.847390 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4081fd8-45fb-425a-9d49-287d18032e8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:33.848805 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4081fd8-45fb-425a-9d49-287d18032e8f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:33.857469 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 59/120] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:33.862451 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4081fd8-45fb-425a-9d49-287d18032e8f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:33.908133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:33.915681 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:33.931995 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5d0ed-627c-4bb5-9324-947cea91e243 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:33.969033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6344b-9529-41aa-a11e-df1f9eeca01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:33.969647 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6344b-9529-41aa-a11e-df1f9eeca01b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:33.969890 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4081fd8-45fb-425a-9d49-287d18032e8f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:33.970022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6344b-9529-41aa-a11e-df1f9eeca01b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:33.970192 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6344b-9529-41aa-a11e-df1f9eeca01b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:33.970918 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da6344b-9529-41aa-a11e-df1f9eeca01b None admin] RBAC: Adding query filter params (role.id=78c5ea5c9f154c6da4912cfc338b6474, scope.project.id=9c9a0023662b41afa1d7bec05fd12eda, user.id=9bdcd0c1fbd34f56abe6a3fd9c128c69) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:33.971254 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da6344b-9529-41aa-a11e-df1f9eeca01b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:33.972692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da6344b-9529-41aa-a11e-df1f9eeca01b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:33.978552 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 62/121] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:22:33 2022] GET /identity/v3/role_assignments?user.id=9bdcd0c1fbd34f56abe6a3fd9c128c69&scope.project.id=9c9a0023662b41afa1d7bec05fd12eda&role.id=78c5ea5c9f154c6da4912cfc338b6474 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:33.979379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6344b-9529-41aa-a11e-df1f9eeca01b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:34.018970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da6705-6637-46c4-ab2f-467b25fbf359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:34.020084 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da6705-6637-46c4-ab2f-467b25fbf359 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:34.020679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da6705-6637-46c4-ab2f-467b25fbf359 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:34.021147 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da6705-6637-46c4-ab2f-467b25fbf359 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:34.034912 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da6705-6637-46c4-ab2f-467b25fbf359 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:34.042369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-c9da6705-6637-46c4-ab2f-467b25fbf359 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:22:34.042369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:34.042369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:34.042369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:34.042369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:34.042369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:34.042369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:34.042369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:34.042369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:34.042369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:34.042369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:34.042369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:34.042369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:34.042369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:34.042369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:34.042369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:34.043332 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:34.043332 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:34.043332 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:34.043332 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:34.043332 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:34.043332 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:34.043332 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:34.043332 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:34.043332 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:34.043332 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:34.043332 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:34.043332 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:34.043332 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:34.043332 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:34.043332 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:34.043332 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:34.044090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:34.044090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:34.044090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:34.044090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:34.044090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:34.044090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:34.044090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:34.044090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:34.044090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:34.044090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:34.044090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:22:34.044090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:22:34.046489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 60/122] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 12:22:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:34.047644 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da6705-6637-46c4-ab2f-467b25fbf359 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:34.105970 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6344b-9529-41aa-a11e-df1f9eeca01b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:34.146911 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70fc00-2d8d-4ffd-a430-73ca13129d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:34.147657 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70fc00-2d8d-4ffd-a430-73ca13129d1e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:34.147808 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70fc00-2d8d-4ffd-a430-73ca13129d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:34.148073 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70fc00-2d8d-4ffd-a430-73ca13129d1e None admin] PATH_INFO: `/v3/users/a43f0c8b634b4f2997cbdf44c7dd2c5a` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:34.149255 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e70fc00-2d8d-4ffd-a430-73ca13129d1e None admin] RBAC: Authorizing `identity:get_user(user_id=a43f0c8b634b4f2997cbdf44c7dd2c5a)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:34.150713 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e70fc00-2d8d-4ffd-a430-73ca13129d1e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:34.152542 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 63/123] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:33 2022] GET /identity/v3/users/a43f0c8b634b4f2997cbdf44c7dd2c5a => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:34.153096 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70fc00-2d8d-4ffd-a430-73ca13129d1e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:34.164511 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:34.170468 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da6705-6637-46c4-ab2f-467b25fbf359 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:34.181345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:34.216979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f87ace-6395-41e4-92ea-d1ff9108585f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:34.218136 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f87ace-6395-41e4-92ea-d1ff9108585f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:34.219067 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f87ace-6395-41e4-92ea-d1ff9108585f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:34.219518 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f87ace-6395-41e4-92ea-d1ff9108585f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:34.239692 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f87ace-6395-41e4-92ea-d1ff9108585f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:34.248570 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-27f87ace-6395-41e4-92ea-d1ff9108585f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:22:34.248570 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:34.248570 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:34.248570 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:34.248570 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:34.248570 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:34.248570 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:34.248570 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:34.248570 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:34.248570 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:34.248570 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:34.248570 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:34.248570 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:34.248570 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:34.248570 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:34.248570 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:34.249644 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:34.249644 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:34.249644 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:34.249644 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:34.249644 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:34.249644 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:34.249644 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:34.249644 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:34.249644 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:34.249644 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:34.249644 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:34.249644 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:34.249644 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:34.249644 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:34.249644 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:34.249644 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:34.250564 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:34.250564 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:34.250564 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:34.250564 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:34.250564 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:34.250564 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:34.250564 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:34.250564 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:34.250564 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:34.250564 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:34.250564 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:22:34.250564 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:22:34.253081 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 61/124] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:34.254015 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f87ace-6395-41e4-92ea-d1ff9108585f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:34.276103 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70fc00-2d8d-4ffd-a430-73ca13129d1e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:34.314134 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044cffb-0ca2-4b45-94dc-adb7b20120ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:34.315130 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044cffb-0ca2-4b45-94dc-adb7b20120ae None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:34.315130 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044cffb-0ca2-4b45-94dc-adb7b20120ae None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:34.315338 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044cffb-0ca2-4b45-94dc-adb7b20120ae None admin] PATH_INFO: `/v3/users/903d09eed7334b43af767deea1a6bd32` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:34.316510 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2044cffb-0ca2-4b45-94dc-adb7b20120ae None admin] RBAC: Authorizing `identity:get_user(user_id=903d09eed7334b43af767deea1a6bd32)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:34.317954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2044cffb-0ca2-4b45-94dc-adb7b20120ae None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:34.320922 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 64/125] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:34 2022] GET /identity/v3/users/903d09eed7334b43af767deea1a6bd32 => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:34.321553 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044cffb-0ca2-4b45-94dc-adb7b20120ae None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:34.385977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f87ace-6395-41e4-92ea-d1ff9108585f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:34.409856 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:34.427114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc41bc1-c7b0-48cc-a85f-222abc847ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:34.428143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc41bc1-c7b0-48cc-a85f-222abc847ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:34.428623 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc41bc1-c7b0-48cc-a85f-222abc847ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:34.429091 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc41bc1-c7b0-48cc-a85f-222abc847ed2 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:34.430070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc41bc1-c7b0-48cc-a85f-222abc847ed2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:34.430573 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc41bc1-c7b0-48cc-a85f-222abc847ed2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:34.432347 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc41bc1-c7b0-48cc-a85f-222abc847ed2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:34.435342 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044cffb-0ca2-4b45-94dc-adb7b20120ae None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:34.441741 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 62/126] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:22:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:34.442849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc41bc1-c7b0-48cc-a85f-222abc847ed2 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:34.474475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b81710-10ce-45d2-a93d-35d630d9ef5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:34.475672 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b81710-10ce-45d2-a93d-35d630d9ef5c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:34.476156 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b81710-10ce-45d2-a93d-35d630d9ef5c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:34.476650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b81710-10ce-45d2-a93d-35d630d9ef5c None admin] PATH_INFO: `/v3/projects/9c9a0023662b41afa1d7bec05fd12eda` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:34.477998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b81710-10ce-45d2-a93d-35d630d9ef5c None admin] RBAC: Authorizing `identity:get_project(project_id=9c9a0023662b41afa1d7bec05fd12eda)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:34.478100 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:34.479773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b81710-10ce-45d2-a93d-35d630d9ef5c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:34.481593 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 65/127] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:22:34 2022] GET /identity/v3/projects/9c9a0023662b41afa1d7bec05fd12eda => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:34.482539 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b81710-10ce-45d2-a93d-35d630d9ef5c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:34.571914 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc41bc1-c7b0-48cc-a85f-222abc847ed2 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:34.599311 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b81710-10ce-45d2-a93d-35d630d9ef5c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:34.616399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06353cff-d948-41bf-b641-3f3e9c8ede4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:34.617190 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06353cff-d948-41bf-b641-3f3e9c8ede4f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:34.617636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06353cff-d948-41bf-b641-3f3e9c8ede4f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:34.617918 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06353cff-d948-41bf-b641-3f3e9c8ede4f None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:34.618985 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06353cff-d948-41bf-b641-3f3e9c8ede4f None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:34.619335 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06353cff-d948-41bf-b641-3f3e9c8ede4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:34.620979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06353cff-d948-41bf-b641-3f3e9c8ede4f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:34.632705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 63/128] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 12:22:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:34.633616 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06353cff-d948-41bf-b641-3f3e9c8ede4f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:34.636704 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61dfcc-8068-447b-b031-5aea3e88574a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:34.637351 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61dfcc-8068-447b-b031-5aea3e88574a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:34.637640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61dfcc-8068-447b-b031-5aea3e88574a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:34.637921 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61dfcc-8068-447b-b031-5aea3e88574a None admin] PATH_INFO: `/v3/projects/9c9a0023662b41afa1d7bec05fd12eda` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:34.639080 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61dfcc-8068-447b-b031-5aea3e88574a None admin] RBAC: Authorizing `identity:get_project(project_id=9c9a0023662b41afa1d7bec05fd12eda)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:34.640686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61dfcc-8068-447b-b031-5aea3e88574a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:34.642330 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 66/129] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:22:34 2022] GET /identity/v3/projects/9c9a0023662b41afa1d7bec05fd12eda => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:34.648044 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61dfcc-8068-447b-b031-5aea3e88574a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:34.669259 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:34.738577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:34.750310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06353cff-d948-41bf-b641-3f3e9c8ede4f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:34.764005 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61dfcc-8068-447b-b031-5aea3e88574a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:34.789179 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc95e87-502c-4014-9859-958d1667ea70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:34.790157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc95e87-502c-4014-9859-958d1667ea70 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:34.790617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc95e87-502c-4014-9859-958d1667ea70 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:34.791037 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc95e87-502c-4014-9859-958d1667ea70 None admin] PATH_INFO: `/v3/users/a43f0c8b634b4f2997cbdf44c7dd2c5a` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:34.792451 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc95e87-502c-4014-9859-958d1667ea70 None admin] RBAC: Authorizing `identity:get_user(user_id=a43f0c8b634b4f2997cbdf44c7dd2c5a)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:34.794100 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc95e87-502c-4014-9859-958d1667ea70 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:34.796105 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 64/130] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:34 2022] GET /identity/v3/users/a43f0c8b634b4f2997cbdf44c7dd2c5a => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:34.797162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc95e87-502c-4014-9859-958d1667ea70 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:34.801065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574a914-79f8-4a85-b1f8-3e076dee53b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:34.801994 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574a914-79f8-4a85-b1f8-3e076dee53b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:34.801994 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574a914-79f8-4a85-b1f8-3e076dee53b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:34.802274 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574a914-79f8-4a85-b1f8-3e076dee53b7 None admin] PATH_INFO: `/v3/projects/9c9a0023662b41afa1d7bec05fd12eda/users/a43f0c8b634b4f2997cbdf44c7dd2c5a/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:34.804435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a574a914-79f8-4a85-b1f8-3e076dee53b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c9a0023662b41afa1d7bec05fd12eda, user_id=a43f0c8b634b4f2997cbdf44c7dd2c5a, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:34.806469 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a574a914-79f8-4a85-b1f8-3e076dee53b7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:34.882521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 67/131] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 12:22:34 2022] PUT /identity/v3/projects/9c9a0023662b41afa1d7bec05fd12eda/users/a43f0c8b634b4f2997cbdf44c7dd2c5a/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:22:34.887291 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574a914-79f8-4a85-b1f8-3e076dee53b7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:34.950262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc95e87-502c-4014-9859-958d1667ea70 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:34.983637 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:35.010904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574a914-79f8-4a85-b1f8-3e076dee53b7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:35.053478 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9779f3-0384-4008-962d-d4c5818dbb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:35.054239 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9779f3-0384-4008-962d-d4c5818dbb63 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:35.054527 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9779f3-0384-4008-962d-d4c5818dbb63 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:35.054814 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9779f3-0384-4008-962d-d4c5818dbb63 None admin] PATH_INFO: `/v3/projects/9c9a0023662b41afa1d7bec05fd12eda/users/903d09eed7334b43af767deea1a6bd32/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:35.056838 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06cff09-9d26-4991-958c-2c62aea3a367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:35.057291 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9779f3-0384-4008-962d-d4c5818dbb63 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c9a0023662b41afa1d7bec05fd12eda, user_id=903d09eed7334b43af767deea1a6bd32, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:35.057498 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cff09-9d26-4991-958c-2c62aea3a367 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:35.057782 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cff09-9d26-4991-958c-2c62aea3a367 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:35.058061 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cff09-9d26-4991-958c-2c62aea3a367 None admin] PATH_INFO: `/v3/users/a43f0c8b634b4f2997cbdf44c7dd2c5a` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:35.058381 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9779f3-0384-4008-962d-d4c5818dbb63 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:35.059255 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06cff09-9d26-4991-958c-2c62aea3a367 None admin] RBAC: Authorizing `identity:get_user(user_id=a43f0c8b634b4f2997cbdf44c7dd2c5a)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:35.060717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06cff09-9d26-4991-958c-2c62aea3a367 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:35.062931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 65/132] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:34 2022] GET /identity/v3/users/a43f0c8b634b4f2997cbdf44c7dd2c5a => generated 275 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:35.063935 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06cff09-9d26-4991-958c-2c62aea3a367 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:35.100690 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:35.164245 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 68/133] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 12:22:34 2022] PUT /identity/v3/projects/9c9a0023662b41afa1d7bec05fd12eda/users/903d09eed7334b43af767deea1a6bd32/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:22:35.164916 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9779f3-0384-4008-962d-d4c5818dbb63 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:35.186516 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06cff09-9d26-4991-958c-2c62aea3a367 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:35.244265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e065f0ff-1bc7-4cb7-b1b9-f7c09239c82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:35.245150 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065f0ff-1bc7-4cb7-b1b9-f7c09239c82e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:35.245440 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065f0ff-1bc7-4cb7-b1b9-f7c09239c82e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:35.245722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065f0ff-1bc7-4cb7-b1b9-f7c09239c82e None admin] PATH_INFO: `/v3/projects/9c9a0023662b41afa1d7bec05fd12eda` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:35.249193 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e065f0ff-1bc7-4cb7-b1b9-f7c09239c82e None admin] RBAC: Authorizing `identity:get_project(project_id=9c9a0023662b41afa1d7bec05fd12eda)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:35.250661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e065f0ff-1bc7-4cb7-b1b9-f7c09239c82e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:35.252511 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 66/134] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:22:35 2022] GET /identity/v3/projects/9c9a0023662b41afa1d7bec05fd12eda => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:35.259471 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e065f0ff-1bc7-4cb7-b1b9-f7c09239c82e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:35.313275 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9779f3-0384-4008-962d-d4c5818dbb63 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:35.348521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52391b-cb40-4b50-8c04-95a3a99159c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:35.349189 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52391b-cb40-4b50-8c04-95a3a99159c1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:35.349474 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52391b-cb40-4b50-8c04-95a3a99159c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:35.349753 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52391b-cb40-4b50-8c04-95a3a99159c1 None admin] PATH_INFO: `/v3/projects/f9ec7e64955644058c55670463303dd3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:35.351685 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52391b-cb40-4b50-8c04-95a3a99159c1 None admin] RBAC: Authorizing `identity:get_project(project_id=f9ec7e64955644058c55670463303dd3)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:35.353276 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52391b-cb40-4b50-8c04-95a3a99159c1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:35.354981 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 69/135] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:22:35 2022] GET /identity/v3/projects/f9ec7e64955644058c55670463303dd3 => generated 312 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:35.355509 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:35.360818 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52391b-cb40-4b50-8c04-95a3a99159c1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:35.410313 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:35.419077 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e065f0ff-1bc7-4cb7-b1b9-f7c09239c82e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:35.500074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48821242-3c30-4926-8d75-cb9ef7f47b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:35.500811 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48821242-3c30-4926-8d75-cb9ef7f47b90 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:35.501101 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48821242-3c30-4926-8d75-cb9ef7f47b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:35.501385 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48821242-3c30-4926-8d75-cb9ef7f47b90 None admin] PATH_INFO: `/v3/projects/9c9a0023662b41afa1d7bec05fd12eda/users/a43f0c8b634b4f2997cbdf44c7dd2c5a/roles/0f58d2782a2840648bbbc2ab26ec02df` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:35.508197 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48821242-3c30-4926-8d75-cb9ef7f47b90 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c9a0023662b41afa1d7bec05fd12eda, user_id=a43f0c8b634b4f2997cbdf44c7dd2c5a, role_id=0f58d2782a2840648bbbc2ab26ec02df)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:35.509690 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48821242-3c30-4926-8d75-cb9ef7f47b90 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:35.519261 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52391b-cb40-4b50-8c04-95a3a99159c1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:35.558455 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d713d29-d947-48e7-859d-0af53b6bcb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:35.559410 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d713d29-d947-48e7-859d-0af53b6bcb02 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:35.559877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d713d29-d947-48e7-859d-0af53b6bcb02 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:35.560299 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d713d29-d947-48e7-859d-0af53b6bcb02 None admin] PATH_INFO: `/v3/projects/f9ec7e64955644058c55670463303dd3/users/a43f0c8b634b4f2997cbdf44c7dd2c5a/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:35.562579 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d713d29-d947-48e7-859d-0af53b6bcb02 None admin] RBAC: Authorizing `identity:create_grant(project_id=f9ec7e64955644058c55670463303dd3, user_id=a43f0c8b634b4f2997cbdf44c7dd2c5a, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:35.564229 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d713d29-d947-48e7-859d-0af53b6bcb02 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:35.596168 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 67/136] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 12:22:35 2022] PUT /identity/v3/projects/9c9a0023662b41afa1d7bec05fd12eda/users/a43f0c8b634b4f2997cbdf44c7dd2c5a/roles/0f58d2782a2840648bbbc2ab26ec02df => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:22:35.597931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15181ed-edc0-4b55-a0f5-a1aafd0ec7d8 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:35.598413 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15181ed-edc0-4b55-a0f5-a1aafd0ec7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:35.598835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15181ed-edc0-4b55-a0f5-a1aafd0ec7d8 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:35.600694 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 68/137] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:35.606994 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964db51-f31a-42a0-aa30-deea0d38c0f9 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:35.607499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964db51-f31a-42a0-aa30-deea0d38c0f9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:35.608065 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964db51-f31a-42a0-aa30-deea0d38c0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:35.637984 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3964db51-f31a-42a0-aa30-deea0d38c0f9 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:35.639569 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3964db51-f31a-42a0-aa30-deea0d38c0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:35.665570 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 70/138] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 12:22:35 2022] PUT /identity/v3/projects/f9ec7e64955644058c55670463303dd3/users/a43f0c8b634b4f2997cbdf44c7dd2c5a/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:22:35.741768 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 69/139] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:35.779673 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d713d29-d947-48e7-859d-0af53b6bcb02 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:35.856262 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:35.910360 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d713d29-d947-48e7-859d-0af53b6bcb02 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:35.949026 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b648936d-8fe8-4863-baad-92dca940b33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:35.949679 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648936d-8fe8-4863-baad-92dca940b33d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:35.950013 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648936d-8fe8-4863-baad-92dca940b33d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:35.950299 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648936d-8fe8-4863-baad-92dca940b33d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:35.951873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b648936d-8fe8-4863-baad-92dca940b33d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:35.953378 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b648936d-8fe8-4863-baad-92dca940b33d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:35.955102 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 71/140] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:22:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:35.961269 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964db51-f31a-42a0-aa30-deea0d38c0f9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:36.066217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:36.082490 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964db51-f31a-42a0-aa30-deea0d38c0f9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:36.118982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd1d51-a799-41a2-93d8-e9eb207c9199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:36.119652 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd1d51-a799-41a2-93d8-e9eb207c9199 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:36.120190 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd1d51-a799-41a2-93d8-e9eb207c9199 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:36.120472 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd1d51-a799-41a2-93d8-e9eb207c9199 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:36.121124 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd1d51-a799-41a2-93d8-e9eb207c9199 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:36.122600 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd1d51-a799-41a2-93d8-e9eb207c9199 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:36.252634 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 70/141] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 12:22:35 2022] POST /identity/v3/users => generated 283 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:22:36.374283 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07311fe8-9e47-41e2-b737-58dd7ce27262 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:36.374769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07311fe8-9e47-41e2-b737-58dd7ce27262 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:36.374841 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07311fe8-9e47-41e2-b737-58dd7ce27262 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:36.376336 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 72/142] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:36.386440 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89ffdd-2bc9-4e99-a4ca-f25d6e65dce6 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:36.386759 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89ffdd-2bc9-4e99-a4ca-f25d6e65dce6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:36.387000 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89ffdd-2bc9-4e99-a4ca-f25d6e65dce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:36.412385 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-da89ffdd-2bc9-4e99-a4ca-f25d6e65dce6 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:36.414166 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-da89ffdd-2bc9-4e99-a4ca-f25d6e65dce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:36.507012 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 71/143] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:36.552567 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07311fe8-9e47-41e2-b737-58dd7ce27262 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:36.671220 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07311fe8-9e47-41e2-b737-58dd7ce27262 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:36.699498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:36.708362 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ff905-2c09-4937-8d01-468f26ce0b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:36.709002 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ff905-2c09-4937-8d01-468f26ce0b26 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:36.709492 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ff905-2c09-4937-8d01-468f26ce0b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:36.709778 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ff905-2c09-4937-8d01-468f26ce0b26 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:36.720443 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ff905-2c09-4937-8d01-468f26ce0b26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:36.726053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-758ff905-2c09-4937-8d01-468f26ce0b26 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:22:36.726053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:36.726053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:36.726053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:36.726053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:36.726053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:36.726053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:36.726053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:36.726053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:36.726053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:36.726053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:36.726053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:36.726053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:36.726053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:36.726053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:36.726053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:36.731979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:36.731979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:36.731979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:36.731979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:36.731979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:36.731979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:36.731979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:36.731979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:36.731979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:36.731979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:36.731979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:36.731979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:36.731979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:36.731979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:36.731979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:36.731979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:36.732723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:36.732723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:36.732723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:36.732723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:36.732723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:36.732723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:36.732723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:36.732723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:36.732723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:36.732723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:36.732723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:22:36.732723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:22:36.732723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 73/144] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:36.733832 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89ffdd-2bc9-4e99-a4ca-f25d6e65dce6 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:36.809849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:36.842874 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89ffdd-2bc9-4e99-a4ca-f25d6e65dce6 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:36.879088 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eaa992-fc7a-4944-8761-c18e579e3b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:36.879745 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eaa992-fc7a-4944-8761-c18e579e3b42 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:36.880030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eaa992-fc7a-4944-8761-c18e579e3b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:36.880306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eaa992-fc7a-4944-8761-c18e579e3b42 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:36.880979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eaa992-fc7a-4944-8761-c18e579e3b42 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:36.881829 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eaa992-fc7a-4944-8761-c18e579e3b42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:36.883415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eaa992-fc7a-4944-8761-c18e579e3b42 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:36.898527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 72/145] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 12:22:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:36.900316 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ff905-2c09-4937-8d01-468f26ce0b26 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:37.029302 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ff905-2c09-4937-8d01-468f26ce0b26 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:37.070754 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee8678-6cdd-4ead-9a8f-a1bd2adf9e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:37.071798 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee8678-6cdd-4ead-9a8f-a1bd2adf9e10 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:37.072360 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee8678-6cdd-4ead-9a8f-a1bd2adf9e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:37.072833 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee8678-6cdd-4ead-9a8f-a1bd2adf9e10 None admin] PATH_INFO: `/v3/users/9bdcd0c1fbd34f56abe6a3fd9c128c69` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:37.074187 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee8678-6cdd-4ead-9a8f-a1bd2adf9e10 None admin] RBAC: Authorizing `identity:get_user(user_id=9bdcd0c1fbd34f56abe6a3fd9c128c69)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:37.075851 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee8678-6cdd-4ead-9a8f-a1bd2adf9e10 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:37.077733 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 74/146] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:36 2022] GET /identity/v3/users/9bdcd0c1fbd34f56abe6a3fd9c128c69 => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:37.083551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eaa992-fc7a-4944-8761-c18e579e3b42 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:37.211058 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eaa992-fc7a-4944-8761-c18e579e3b42 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:37.242031 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:37.250694 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f415ca38-0c4d-4152-92bc-aea72fd1ce7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:37.251384 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415ca38-0c4d-4152-92bc-aea72fd1ce7e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:37.251731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415ca38-0c4d-4152-92bc-aea72fd1ce7e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:37.252011 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415ca38-0c4d-4152-92bc-aea72fd1ce7e None admin] PATH_INFO: `/v3/projects/9c9a0023662b41afa1d7bec05fd12eda` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:37.253160 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f415ca38-0c4d-4152-92bc-aea72fd1ce7e None admin] RBAC: Authorizing `identity:get_project(project_id=9c9a0023662b41afa1d7bec05fd12eda)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:37.254605 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f415ca38-0c4d-4152-92bc-aea72fd1ce7e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:37.256905 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 73/147] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:22:37 2022] GET /identity/v3/projects/9c9a0023662b41afa1d7bec05fd12eda => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:37.263091 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee8678-6cdd-4ead-9a8f-a1bd2adf9e10 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:37.335854 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:37.395083 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee8678-6cdd-4ead-9a8f-a1bd2adf9e10 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:37.433809 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73cd9c-31c3-4942-98df-5ebda835e939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:37.434478 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73cd9c-31c3-4942-98df-5ebda835e939 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:37.434759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73cd9c-31c3-4942-98df-5ebda835e939 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:37.435030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73cd9c-31c3-4942-98df-5ebda835e939 None admin] PATH_INFO: `/v3/projects/9c9a0023662b41afa1d7bec05fd12eda/users/9bdcd0c1fbd34f56abe6a3fd9c128c69/roles/78c5ea5c9f154c6da4912cfc338b6474` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:37.437703 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73cd9c-31c3-4942-98df-5ebda835e939 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c9a0023662b41afa1d7bec05fd12eda, user_id=9bdcd0c1fbd34f56abe6a3fd9c128c69, role_id=78c5ea5c9f154c6da4912cfc338b6474)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:37.439193 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73cd9c-31c3-4942-98df-5ebda835e939 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:37.517839 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 75/148] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 12:22:37 2022] PUT /identity/v3/projects/9c9a0023662b41afa1d7bec05fd12eda/users/9bdcd0c1fbd34f56abe6a3fd9c128c69/roles/78c5ea5c9f154c6da4912cfc338b6474 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:22:38.097800 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2afda-45fa-4b60-bfc3-b5df2b670e85 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:38.098335 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2afda-45fa-4b60-bfc3-b5df2b670e85 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:38.098335 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2afda-45fa-4b60-bfc3-b5df2b670e85 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:38.100674 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 74/149] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:38.108009 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea27e6-4eea-410a-a4c6-5be745c0979e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:38.108300 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea27e6-4eea-410a-a4c6-5be745c0979e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:38.108579 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea27e6-4eea-410a-a4c6-5be745c0979e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:38.128346 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f4ea27e6-4eea-410a-a4c6-5be745c0979e None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:38.129616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f4ea27e6-4eea-410a-a4c6-5be745c0979e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:38.211498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 76/150] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:22:38.249070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2afda-45fa-4b60-bfc3-b5df2b670e85 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:38.384563 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2afda-45fa-4b60-bfc3-b5df2b670e85 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:38.422803 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20dcef-e726-4039-918d-7e96b51b384e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:38.423555 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20dcef-e726-4039-918d-7e96b51b384e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:38.423840 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20dcef-e726-4039-918d-7e96b51b384e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:38.424116 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20dcef-e726-4039-918d-7e96b51b384e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:38.438475 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20dcef-e726-4039-918d-7e96b51b384e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:38.443390 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-9c20dcef-e726-4039-918d-7e96b51b384e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:22:38.443390 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:38.443390 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:38.443390 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:38.443390 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:38.443390 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:38.443390 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:38.443390 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:38.443390 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:38.443390 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:38.443390 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:38.443390 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:38.443390 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:38.443390 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:38.443390 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:38.443390 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:38.444871 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:38.444871 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:38.444871 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:38.444871 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:38.444871 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:38.444871 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:38.444871 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:38.444871 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:38.444871 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:38.444871 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:38.444871 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:38.444871 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:38.444871 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:38.444871 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:38.444871 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:38.444871 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:38.445661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:38.445661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:38.445661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:38.445661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:38.445661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:38.445661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:38.445661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:38.445661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:38.445661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:38.445661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:38.445661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:22:38.445661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:22:38.449261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 75/151] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:38.452212 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea27e6-4eea-410a-a4c6-5be745c0979e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:38.501932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:38.557960 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea27e6-4eea-410a-a4c6-5be745c0979e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:38.594495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403fc60a-c173-458c-a419-25b9f31c6bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:38.595113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403fc60a-c173-458c-a419-25b9f31c6bda None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:38.595426 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403fc60a-c173-458c-a419-25b9f31c6bda None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:38.595704 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403fc60a-c173-458c-a419-25b9f31c6bda None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:38.596378 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403fc60a-c173-458c-a419-25b9f31c6bda None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:38.596684 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403fc60a-c173-458c-a419-25b9f31c6bda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:38.598061 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403fc60a-c173-458c-a419-25b9f31c6bda None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:38.610590 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 77/152] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 12:22:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:38.613165 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20dcef-e726-4039-918d-7e96b51b384e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:38.628807 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914acb9-09d5-4e5c-965f-fb7cd2bf6307 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:38.629097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914acb9-09d5-4e5c-965f-fb7cd2bf6307 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:38.629363 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914acb9-09d5-4e5c-965f-fb7cd2bf6307 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:38.630778 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 78/153] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:38.637903 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9c019-327e-421c-b2bb-d88429d82c2e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:38.638129 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9c019-327e-421c-b2bb-d88429d82c2e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:38.638378 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9c019-327e-421c-b2bb-d88429d82c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:38.669011 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f3b9c019-327e-421c-b2bb-d88429d82c2e None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:38.670232 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f3b9c019-327e-421c-b2bb-d88429d82c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:38.731215 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:38.800682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 79/154] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:22:38.803924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20dcef-e726-4039-918d-7e96b51b384e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:38.847944 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b9c019-327e-421c-b2bb-d88429d82c2e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:38.856825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd3dac-b511-4755-9e0a-724e5526dec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:38.857475 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd3dac-b511-4755-9e0a-724e5526dec6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:38.857756 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd3dac-b511-4755-9e0a-724e5526dec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:38.858031 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd3dac-b511-4755-9e0a-724e5526dec6 None admin] PATH_INFO: `/v3/users/a43f0c8b634b4f2997cbdf44c7dd2c5a` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:38.860711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd3dac-b511-4755-9e0a-724e5526dec6 None admin] RBAC: Authorizing `identity:get_user(user_id=a43f0c8b634b4f2997cbdf44c7dd2c5a)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:38.862675 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd3dac-b511-4755-9e0a-724e5526dec6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:38.864762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 76/155] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:38 2022] GET /identity/v3/users/a43f0c8b634b4f2997cbdf44c7dd2c5a => generated 275 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:38.871620 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd3dac-b511-4755-9e0a-724e5526dec6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:38.949980 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:39.054466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd3dac-b511-4755-9e0a-724e5526dec6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:39.106518 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b9c019-327e-421c-b2bb-d88429d82c2e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:39.118204 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:39.137097 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c722d7-d526-4d75-9726-04349b9b2f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:39.138968 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c722d7-d526-4d75-9726-04349b9b2f0e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:39.138968 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c722d7-d526-4d75-9726-04349b9b2f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:39.138968 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c722d7-d526-4d75-9726-04349b9b2f0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:39.183439 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc171146-9baa-4dad-a702-6fb631578043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:39.183996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc171146-9baa-4dad-a702-6fb631578043 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:39.184285 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc171146-9baa-4dad-a702-6fb631578043 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:39.184563 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc171146-9baa-4dad-a702-6fb631578043 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:39.206165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc171146-9baa-4dad-a702-6fb631578043 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:39.211815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-dc171146-9baa-4dad-a702-6fb631578043 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:22:39.211815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:39.211815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:39.211815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:39.211815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:39.211815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:39.211815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:39.211815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:39.211815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:39.211815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:39.211815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:39.211815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:39.211815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:39.211815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:39.211815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:39.211815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:39.212656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:39.212656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:39.212656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:39.212656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:39.212656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:39.212656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:39.212656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:39.212656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:39.212656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:39.212656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:39.212656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:39.212656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:39.212656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:39.212656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:39.212656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:39.212656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:39.213466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:39.213466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:39.213466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:39.213466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:39.213466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:39.213466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:39.213466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:39.213466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:39.213466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:39.213466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:39.213466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:22:39.213466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:22:39.213466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 80/156] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:22:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:39.216151 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9ce97-9d70-4991-9a3e-016a6d0c45da None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:39.216445 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9ce97-9d70-4991-9a3e-016a6d0c45da None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:39.216728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9ce97-9d70-4991-9a3e-016a6d0c45da None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:39.218056 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 81/157] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:39.221881 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9ce97-9d70-4991-9a3e-016a6d0c45da None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:39.353364 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9ce97-9d70-4991-9a3e-016a6d0c45da None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:39.364065 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c722d7-d526-4d75-9726-04349b9b2f0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:39.366136 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c722d7-d526-4d75-9726-04349b9b2f0e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:39.375624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:39.395110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e8a60-6308-452d-a4b9-54bfe36fed80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:39.395764 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e8a60-6308-452d-a4b9-54bfe36fed80 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:39.396212 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e8a60-6308-452d-a4b9-54bfe36fed80 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:39.396499 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e8a60-6308-452d-a4b9-54bfe36fed80 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:39.397204 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e8a60-6308-452d-a4b9-54bfe36fed80 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:39.397577 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e8a60-6308-452d-a4b9-54bfe36fed80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:39.399059 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e8a60-6308-452d-a4b9-54bfe36fed80 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:39.408883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 82/158] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:39.410191 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397634b1-f805-4976-b5f5-a19fd4b89bca None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:39.410545 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397634b1-f805-4976-b5f5-a19fd4b89bca None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:39.412321 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397634b1-f805-4976-b5f5-a19fd4b89bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:39.431581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-397634b1-f805-4976-b5f5-a19fd4b89bca None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:39.432796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-397634b1-f805-4976-b5f5-a19fd4b89bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:39.507765 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 83/159] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:39.508652 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397634b1-f805-4976-b5f5-a19fd4b89bca None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:39.591010 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 77/160] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:22:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 720 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 12:22:39.592143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef96c5b-0fab-4c4b-910e-72c17def9e86 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:39.592431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef96c5b-0fab-4c4b-910e-72c17def9e86 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:39.592707 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef96c5b-0fab-4c4b-910e-72c17def9e86 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:39.593951 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 78/161] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:39.600074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef96c5b-0fab-4c4b-910e-72c17def9e86 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:39.624858 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397634b1-f805-4976-b5f5-a19fd4b89bca None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:39.627471 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:39.649028 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:39.664226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b14fd7-7ac3-4004-bd11-9d2cd3f56fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:39.664875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b14fd7-7ac3-4004-bd11-9d2cd3f56fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:39.665925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b14fd7-7ac3-4004-bd11-9d2cd3f56fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:39.666205 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b14fd7-7ac3-4004-bd11-9d2cd3f56fc8 None admin] PATH_INFO: `/v3/users/903d09eed7334b43af767deea1a6bd32` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:39.671946 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b14fd7-7ac3-4004-bd11-9d2cd3f56fc8 None admin] RBAC: Authorizing `identity:get_user(user_id=903d09eed7334b43af767deea1a6bd32)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:39.673464 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b14fd7-7ac3-4004-bd11-9d2cd3f56fc8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:39.675230 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 84/162] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:39 2022] GET /identity/v3/users/903d09eed7334b43af767deea1a6bd32 => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:39.678850 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154e8bf-6f6d-47de-adbf-b15861e3467d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:39.679140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154e8bf-6f6d-47de-adbf-b15861e3467d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:39.691236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154e8bf-6f6d-47de-adbf-b15861e3467d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:39.711740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0154e8bf-6f6d-47de-adbf-b15861e3467d None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:39.712959 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0154e8bf-6f6d-47de-adbf-b15861e3467d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:39.778743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef96c5b-0fab-4c4b-910e-72c17def9e86 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:39.833983 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96229d-f320-4b40-baea-eeafc45ff592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:39.835205 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96229d-f320-4b40-baea-eeafc45ff592 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:39.835516 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96229d-f320-4b40-baea-eeafc45ff592 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:39.835758 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96229d-f320-4b40-baea-eeafc45ff592 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:39.838277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 85/163] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:39.838963 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154e8bf-6f6d-47de-adbf-b15861e3467d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:39.861027 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96229d-f320-4b40-baea-eeafc45ff592 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:39.866593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-4d96229d-f320-4b40-baea-eeafc45ff592 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:22:39.866593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:39.866593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:39.866593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:39.866593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:39.866593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:39.866593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:39.866593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:39.866593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:39.866593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:39.866593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:39.866593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:39.866593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:39.866593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:39.866593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:39.866593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:39.869130 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:39.869130 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:39.869130 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:39.869130 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:39.869130 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:39.869130 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:39.869130 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:39.869130 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:39.869130 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:39.869130 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:39.869130 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:39.869130 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:39.869130 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:39.869130 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:39.869130 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:39.869130 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:39.871509 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:39.871509 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:39.871509 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:39.871509 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:39.871509 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:39.871509 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:39.871509 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:39.871509 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:39.871509 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:39.871509 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:39.871509 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:22:39.871509 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:22:39.871509 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 79/164] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:39.871509 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96229d-f320-4b40-baea-eeafc45ff592 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:39.971543 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:39.980562 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154e8bf-6f6d-47de-adbf-b15861e3467d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:40.017114 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f52944e-1b3d-457f-9905-92f7532ac094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:40.019521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52944e-1b3d-457f-9905-92f7532ac094 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:40.019521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52944e-1b3d-457f-9905-92f7532ac094 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:40.019521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52944e-1b3d-457f-9905-92f7532ac094 None admin] PATH_INFO: `/v3/projects/9c9a0023662b41afa1d7bec05fd12eda` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:40.019989 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f52944e-1b3d-457f-9905-92f7532ac094 None admin] RBAC: Authorizing `identity:get_project(project_id=9c9a0023662b41afa1d7bec05fd12eda)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:40.021174 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f52944e-1b3d-457f-9905-92f7532ac094 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:40.023912 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 86/165] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:22:39 2022] GET /identity/v3/projects/9c9a0023662b41afa1d7bec05fd12eda => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:40.024703 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f52944e-1b3d-457f-9905-92f7532ac094 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:40.078054 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:40.100885 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96229d-f320-4b40-baea-eeafc45ff592 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:40.136264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f52944e-1b3d-457f-9905-92f7532ac094 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:40.170946 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0243cfd2-cc8d-4784-8365-81203f7e31a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:40.171594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243cfd2-cc8d-4784-8365-81203f7e31a4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:40.171874 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243cfd2-cc8d-4784-8365-81203f7e31a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:40.172165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243cfd2-cc8d-4784-8365-81203f7e31a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:40.172813 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0243cfd2-cc8d-4784-8365-81203f7e31a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:40.173119 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0243cfd2-cc8d-4784-8365-81203f7e31a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:40.174573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0243cfd2-cc8d-4784-8365-81203f7e31a4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:40.175443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01264d48-282e-4924-9a8a-774a57fdbcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:40.176604 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01264d48-282e-4924-9a8a-774a57fdbcd5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:40.177071 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01264d48-282e-4924-9a8a-774a57fdbcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:40.177541 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01264d48-282e-4924-9a8a-774a57fdbcd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:40.187293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 87/166] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 12:22:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:40.187293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0243cfd2-cc8d-4784-8365-81203f7e31a4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:40.207969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:40.299263 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0243cfd2-cc8d-4784-8365-81203f7e31a4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:40.335574 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b37d9-db41-4f05-a9c2-f7806a762e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:40.336344 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b37d9-db41-4f05-a9c2-f7806a762e3c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:40.336697 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b37d9-db41-4f05-a9c2-f7806a762e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:40.337043 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b37d9-db41-4f05-a9c2-f7806a762e3c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:40.347654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b37d9-db41-4f05-a9c2-f7806a762e3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:40.353199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-718b37d9-db41-4f05-a9c2-f7806a762e3c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:22:40.353199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:40.353199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:40.353199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:40.353199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:40.353199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:40.353199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:40.353199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:40.353199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:40.353199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:40.353199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:40.353199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:40.353199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:40.353199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:40.353199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:40.353199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:40.354053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:40.354053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:40.354053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:40.354053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:40.354053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:40.354053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:40.354053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:40.354053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:40.354053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:40.354053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:40.354053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:40.354053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:40.354053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:40.354053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:40.354053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:40.354053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:40.354915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:40.354915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:40.354915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:40.354915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:40.354915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:40.354915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:40.354915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:40.354915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:40.354915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:40.354915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:40.354915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:22:40.354915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:22:40.354915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 88/167] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 12:22:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:40.355596 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b37d9-db41-4f05-a9c2-f7806a762e3c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:40.357267 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01264d48-282e-4924-9a8a-774a57fdbcd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:40.358992 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01264d48-282e-4924-9a8a-774a57fdbcd5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:40.474366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:40.487207 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b37d9-db41-4f05-a9c2-f7806a762e3c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:40.523880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb39735-486d-4f9d-8f64-c05aa646ebc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:40.524500 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb39735-486d-4f9d-8f64-c05aa646ebc4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:40.524815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb39735-486d-4f9d-8f64-c05aa646ebc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:40.525061 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb39735-486d-4f9d-8f64-c05aa646ebc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:40.525889 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb39735-486d-4f9d-8f64-c05aa646ebc4 None admin] RBAC: Adding query filter params (role.id=872ba0c4212a4684b539a68fc2c20b25, scope.project.id=9c9a0023662b41afa1d7bec05fd12eda, user.id=a43f0c8b634b4f2997cbdf44c7dd2c5a) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:40.526203 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb39735-486d-4f9d-8f64-c05aa646ebc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:40.527706 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb39735-486d-4f9d-8f64-c05aa646ebc4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:40.533931 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 89/168] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:22:40 2022] GET /identity/v3/role_assignments?user.id=a43f0c8b634b4f2997cbdf44c7dd2c5a&scope.project.id=9c9a0023662b41afa1d7bec05fd12eda&role.id=872ba0c4212a4684b539a68fc2c20b25 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:40.534748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb39735-486d-4f9d-8f64-c05aa646ebc4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:40.585698 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:40.622125 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 80/169] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:22:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 754 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:22:40.623293 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01264d48-282e-4924-9a8a-774a57fdbcd5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:40.647627 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb39735-486d-4f9d-8f64-c05aa646ebc4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:40.700332 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d5cd2-58bb-41f0-b0a8-e7675fd5d6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:40.701284 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d5cd2-58bb-41f0-b0a8-e7675fd5d6d6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:40.701740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d5cd2-58bb-41f0-b0a8-e7675fd5d6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:40.702182 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d5cd2-58bb-41f0-b0a8-e7675fd5d6d6 None admin] PATH_INFO: `/v3/users/a43f0c8b634b4f2997cbdf44c7dd2c5a` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:40.703689 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d5cd2-58bb-41f0-b0a8-e7675fd5d6d6 None admin] RBAC: Authorizing `identity:get_user(user_id=a43f0c8b634b4f2997cbdf44c7dd2c5a)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:40.705459 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d5cd2-58bb-41f0-b0a8-e7675fd5d6d6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:40.707805 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 90/170] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:40 2022] GET /identity/v3/users/a43f0c8b634b4f2997cbdf44c7dd2c5a => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:40.708935 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d5cd2-58bb-41f0-b0a8-e7675fd5d6d6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:40.747641 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:40.818391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d5cd2-58bb-41f0-b0a8-e7675fd5d6d6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:40.858150 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ec9bc-bb66-418d-a9fc-85d3b3de6beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:40.859199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ec9bc-bb66-418d-a9fc-85d3b3de6beb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:40.860466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ec9bc-bb66-418d-a9fc-85d3b3de6beb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:40.861622 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ec9bc-bb66-418d-a9fc-85d3b3de6beb None admin] PATH_INFO: `/v3/projects/9c9a0023662b41afa1d7bec05fd12eda` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:40.864291 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041ec9bc-bb66-418d-a9fc-85d3b3de6beb None admin] RBAC: Authorizing `identity:get_project(project_id=9c9a0023662b41afa1d7bec05fd12eda)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:40.867101 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041ec9bc-bb66-418d-a9fc-85d3b3de6beb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:40.871376 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 91/171] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:22:40 2022] GET /identity/v3/projects/9c9a0023662b41afa1d7bec05fd12eda => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:40.873497 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01264d48-282e-4924-9a8a-774a57fdbcd5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:40.875987 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ec9bc-bb66-418d-a9fc-85d3b3de6beb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:40.923296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabcb68-5c07-4fb4-bc49-b44296742d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:40.924250 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabcb68-5c07-4fb4-bc49-b44296742d6a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:40.924713 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabcb68-5c07-4fb4-bc49-b44296742d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:40.925265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabcb68-5c07-4fb4-bc49-b44296742d6a None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:40.926203 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabcb68-5c07-4fb4-bc49-b44296742d6a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:40.926691 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabcb68-5c07-4fb4-bc49-b44296742d6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:40.928917 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabcb68-5c07-4fb4-bc49-b44296742d6a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:40.946965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 81/172] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:22:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:40.948218 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabcb68-5c07-4fb4-bc49-b44296742d6a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:40.982726 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:41.006671 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ec9bc-bb66-418d-a9fc-85d3b3de6beb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:41.045839 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e590732-9ae7-4f8a-be2e-87faf1e31989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:41.046585 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e590732-9ae7-4f8a-be2e-87faf1e31989 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:41.046941 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e590732-9ae7-4f8a-be2e-87faf1e31989 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:41.047416 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e590732-9ae7-4f8a-be2e-87faf1e31989 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:41.051108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:41.123617 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e590732-9ae7-4f8a-be2e-87faf1e31989 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:41.125834 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e590732-9ae7-4f8a-be2e-87faf1e31989 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:41.160349 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabcb68-5c07-4fb4-bc49-b44296742d6a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:41.213740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:41.237170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 92/173] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:22:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:22:41.238157 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e590732-9ae7-4f8a-be2e-87faf1e31989 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:41.250527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb3b950-f8be-4948-8fa3-542f8346c213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:41.251170 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb3b950-f8be-4948-8fa3-542f8346c213 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:41.251460 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb3b950-f8be-4948-8fa3-542f8346c213 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:41.251735 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb3b950-f8be-4948-8fa3-542f8346c213 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:41.252572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb3b950-f8be-4948-8fa3-542f8346c213 None admin] RBAC: Adding query filter params (role.id=bffbe227cda1418cbb58394ddff6f861, scope.project.id=9c9a0023662b41afa1d7bec05fd12eda, user.id=903d09eed7334b43af767deea1a6bd32) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:41.252888 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb3b950-f8be-4948-8fa3-542f8346c213 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:41.254493 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb3b950-f8be-4948-8fa3-542f8346c213 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:41.261436 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 82/174] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:22:40 2022] GET /identity/v3/role_assignments?user.id=903d09eed7334b43af767deea1a6bd32&scope.project.id=9c9a0023662b41afa1d7bec05fd12eda&role.id=bffbe227cda1418cbb58394ddff6f861 => generated 608 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:41.267236 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8fc7a-b158-4ad7-8d0d-5e9b78352b1e None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:41.267236 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8fc7a-b158-4ad7-8d0d-5e9b78352b1e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:41.267236 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8fc7a-b158-4ad7-8d0d-5e9b78352b1e None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:41.267236 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 83/175] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:41.267236 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8ed26-98ce-4111-b177-238619f2b276 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:41.267236 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8ed26-98ce-4111-b177-238619f2b276 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:41.267694 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8ed26-98ce-4111-b177-238619f2b276 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:41.271304 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 84/176] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:41.272817 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8ed26-98ce-4111-b177-238619f2b276 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:41.445339 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8ed26-98ce-4111-b177-238619f2b276 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:41.446245 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e590732-9ae7-4f8a-be2e-87faf1e31989 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:41.469507 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:41.487977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c2dfb-09c2-49bb-855e-cab21bbbd529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:41.489661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c2dfb-09c2-49bb-855e-cab21bbbd529 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:41.490169 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c2dfb-09c2-49bb-855e-cab21bbbd529 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:41.490707 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c2dfb-09c2-49bb-855e-cab21bbbd529 None admin] PATH_INFO: `/v3/projects/f9ec7e64955644058c55670463303dd3` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:41.492897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c2dfb-09c2-49bb-855e-cab21bbbd529 None admin] RBAC: Authorizing `identity:get_project(project_id=f9ec7e64955644058c55670463303dd3)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:41.495529 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c2dfb-09c2-49bb-855e-cab21bbbd529 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:41.498177 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 85/177] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:22:41 2022] GET /identity/v3/projects/f9ec7e64955644058c55670463303dd3 => generated 312 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:41.499640 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133a6c6-34c2-4116-ac68-c444e10b6af4 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:41.500168 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133a6c6-34c2-4116-ac68-c444e10b6af4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:41.500653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133a6c6-34c2-4116-ac68-c444e10b6af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:41.500959 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf0354-ec4c-454f-a3e9-5cd73c8beabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:41.501686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf0354-ec4c-454f-a3e9-5cd73c8beabb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:41.501966 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf0354-ec4c-454f-a3e9-5cd73c8beabb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:41.502234 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf0354-ec4c-454f-a3e9-5cd73c8beabb None admin] PATH_INFO: `/v3/users/a43f0c8b634b4f2997cbdf44c7dd2c5a` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:41.503764 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf0354-ec4c-454f-a3e9-5cd73c8beabb None admin] RBAC: Authorizing `identity:get_user(user_id=a43f0c8b634b4f2997cbdf44c7dd2c5a)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:41.505404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf0354-ec4c-454f-a3e9-5cd73c8beabb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:41.508053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 93/178] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:41 2022] GET /identity/v3/users/a43f0c8b634b4f2997cbdf44c7dd2c5a => generated 275 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:41.509319 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374ef8b-0b5f-4696-8a6b-0b9acc6c3da1 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:41.509606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374ef8b-0b5f-4696-8a6b-0b9acc6c3da1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:41.509862 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374ef8b-0b5f-4696-8a6b-0b9acc6c3da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:41.529437 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6133a6c6-34c2-4116-ac68-c444e10b6af4 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:41.530686 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6133a6c6-34c2-4116-ac68-c444e10b6af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:41.532503 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1374ef8b-0b5f-4696-8a6b-0b9acc6c3da1 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:41.534371 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1374ef8b-0b5f-4696-8a6b-0b9acc6c3da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:41.584703 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:41.606319 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 86/179] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:41.607640 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de6945-cd47-4ef2-9109-658b81c25edb None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:41.607922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de6945-cd47-4ef2-9109-658b81c25edb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:41.608194 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de6945-cd47-4ef2-9109-658b81c25edb None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:41.609466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 87/180] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:41.613926 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 94/181] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:41.614343 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de6945-cd47-4ef2-9109-658b81c25edb None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:41.616528 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374ef8b-0b5f-4696-8a6b-0b9acc6c3da1 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:41.715064 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de6945-cd47-4ef2-9109-658b81c25edb None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:41.717333 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374ef8b-0b5f-4696-8a6b-0b9acc6c3da1 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:41.727673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:41.748586 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef749d7-3191-4548-9c09-1bc988b422ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:41.749198 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef749d7-3191-4548-9c09-1bc988b422ec None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:41.749481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef749d7-3191-4548-9c09-1bc988b422ec None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:41.749758 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef749d7-3191-4548-9c09-1bc988b422ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:41.750480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef749d7-3191-4548-9c09-1bc988b422ec None admin] RBAC: Adding query filter params (role.id=872ba0c4212a4684b539a68fc2c20b25, scope.project.id=f9ec7e64955644058c55670463303dd3, user.id=a43f0c8b634b4f2997cbdf44c7dd2c5a) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:41.750791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef749d7-3191-4548-9c09-1bc988b422ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:41.751515 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9efb813-96c4-4fae-93c7-6594b960caff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:41.752125 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efb813-96c4-4fae-93c7-6594b960caff None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:41.752267 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef749d7-3191-4548-9c09-1bc988b422ec None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:41.752406 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efb813-96c4-4fae-93c7-6594b960caff None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:41.752679 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efb813-96c4-4fae-93c7-6594b960caff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:41.759905 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 88/182] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:22:41 2022] GET /identity/v3/role_assignments?user.id=a43f0c8b634b4f2997cbdf44c7dd2c5a&scope.project.id=f9ec7e64955644058c55670463303dd3&role.id=872ba0c4212a4684b539a68fc2c20b25 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:41.761006 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e2725-2b1f-491d-9e17-2f3303e7a67f None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:41.761413 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e2725-2b1f-491d-9e17-2f3303e7a67f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:41.761696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e2725-2b1f-491d-9e17-2f3303e7a67f None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:41.762945 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 89/183] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:41.767079 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493cc9b2-627c-467f-9bb0-1854a67a6e5d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:41.767079 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493cc9b2-627c-467f-9bb0-1854a67a6e5d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:41.767079 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493cc9b2-627c-467f-9bb0-1854a67a6e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:41.784154 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-493cc9b2-627c-467f-9bb0-1854a67a6e5d None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:41.785627 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-493cc9b2-627c-467f-9bb0-1854a67a6e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:41.809032 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:41.835205 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9efb813-96c4-4fae-93c7-6594b960caff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:41.837246 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9efb813-96c4-4fae-93c7-6594b960caff None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:41.869380 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 90/184] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:41.872821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493cc9b2-627c-467f-9bb0-1854a67a6e5d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:41.952223 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 95/185] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:22:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:22:41.953098 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9efb813-96c4-4fae-93c7-6594b960caff None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:41.996293 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493cc9b2-627c-467f-9bb0-1854a67a6e5d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:42.004341 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:42.032029 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87bc64-3be6-428a-8a9a-c620cb608655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:42.032696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87bc64-3be6-428a-8a9a-c620cb608655 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:42.032979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87bc64-3be6-428a-8a9a-c620cb608655 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:42.033261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87bc64-3be6-428a-8a9a-c620cb608655 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:42.041606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:42.044001 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c87bc64-3be6-428a-8a9a-c620cb608655 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:42.048791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-3c87bc64-3be6-428a-8a9a-c620cb608655 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:22:42.048791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:42.048791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:42.048791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:42.048791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:42.048791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:42.048791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:42.048791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:42.048791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:42.048791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:42.048791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:42.048791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:42.048791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:42.048791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:42.048791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:42.048791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:42.049679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:42.049679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:42.049679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:42.049679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:42.049679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:42.049679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:42.049679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:42.049679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:42.049679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:42.049679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:42.049679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:42.049679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:42.049679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:42.049679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:42.049679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:42.049679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:42.050466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:42.050466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:42.050466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:42.050466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:42.050466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:42.050466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:42.050466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:42.050466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:42.050466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:42.050466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:42.050466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:22:42.050466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:22:42.050466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 91/186] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 12:22:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:42.051097 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfb4b6-2df9-4686-8932-a96b716614b9 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:42.051589 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfb4b6-2df9-4686-8932-a96b716614b9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:42.053685 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfb4b6-2df9-4686-8932-a96b716614b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:42.064420 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9efb813-96c4-4fae-93c7-6594b960caff None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:42.070181 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-59dfb4b6-2df9-4686-8932-a96b716614b9 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:42.071418 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-59dfb4b6-2df9-4686-8932-a96b716614b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:42.099530 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b395b04-c4a1-4ccc-b9f9-98281c54c71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:42.100191 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b395b04-c4a1-4ccc-b9f9-98281c54c71b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:42.100473 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b395b04-c4a1-4ccc-b9f9-98281c54c71b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:42.100747 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b395b04-c4a1-4ccc-b9f9-98281c54c71b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:42.110703 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b395b04-c4a1-4ccc-b9f9-98281c54c71b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:42.115498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-9b395b04-c4a1-4ccc-b9f9-98281c54c71b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:22:42.115498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:42.115498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:42.115498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:42.115498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:42.115498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:42.115498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:42.115498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:42.115498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:42.115498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:42.115498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:42.115498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:42.115498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:42.115498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:42.115498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:42.115498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:42.116486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:42.116486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:42.116486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:42.116486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:42.116486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:42.116486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:42.116486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:42.116486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:42.116486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:42.116486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:42.116486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:42.116486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:42.116486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:42.116486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:42.116486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:42.116486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:42.117487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:42.117487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:42.117487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:42.117487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:42.117487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:42.117487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:42.117487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:42.117487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:42.117487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:42.117487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:42.117487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:22:42.117487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:22:42.117487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 96/187] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:22:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:42.117487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b395b04-c4a1-4ccc-b9f9-98281c54c71b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:42.140654 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 92/188] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:42.141634 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dfb4b6-2df9-4686-8932-a96b716614b9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:42.222022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b395b04-c4a1-4ccc-b9f9-98281c54c71b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:42.237635 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:42.245181 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dfb4b6-2df9-4686-8932-a96b716614b9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:42.258042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0471f1f-abba-4655-854f-abc9b6c47b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:42.258692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0471f1f-abba-4655-854f-abc9b6c47b93 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:42.259141 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0471f1f-abba-4655-854f-abc9b6c47b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:42.259449 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0471f1f-abba-4655-854f-abc9b6c47b93 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:42.260975 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0471f1f-abba-4655-854f-abc9b6c47b93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:42.262421 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0471f1f-abba-4655-854f-abc9b6c47b93 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:42.264186 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 97/189] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:22:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:42.264862 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0471f1f-abba-4655-854f-abc9b6c47b93 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:42.269619 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:42.280679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971017ca-5d0f-4ea1-a878-88b11dd209e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:42.281380 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971017ca-5d0f-4ea1-a878-88b11dd209e0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:42.281656 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971017ca-5d0f-4ea1-a878-88b11dd209e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:42.281924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971017ca-5d0f-4ea1-a878-88b11dd209e0 None admin] PATH_INFO: `/v3/projects/9c9a0023662b41afa1d7bec05fd12eda` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:42.283036 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971017ca-5d0f-4ea1-a878-88b11dd209e0 None admin] RBAC: Authorizing `identity:get_project(project_id=9c9a0023662b41afa1d7bec05fd12eda)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:42.284490 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971017ca-5d0f-4ea1-a878-88b11dd209e0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:42.286101 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 93/190] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:22:42 2022] GET /identity/v3/projects/9c9a0023662b41afa1d7bec05fd12eda => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:42.286849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971017ca-5d0f-4ea1-a878-88b11dd209e0 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:42.365518 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0471f1f-abba-4655-854f-abc9b6c47b93 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:42.387062 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971017ca-5d0f-4ea1-a878-88b11dd209e0 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:42.398269 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcfbcf5-ca92-4d55-a539-703e6559156d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:42.398976 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfbcf5-ca92-4d55-a539-703e6559156d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:42.399290 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfbcf5-ca92-4d55-a539-703e6559156d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:42.399562 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfbcf5-ca92-4d55-a539-703e6559156d None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:42.400216 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcfbcf5-ca92-4d55-a539-703e6559156d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:42.400521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcfbcf5-ca92-4d55-a539-703e6559156d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:42.401892 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcfbcf5-ca92-4d55-a539-703e6559156d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:42.409555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 98/191] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:22:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:42.410329 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcfbcf5-ca92-4d55-a539-703e6559156d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:42.425702 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d3535-d505-4660-9308-d79b35c20bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:42.426317 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d3535-d505-4660-9308-d79b35c20bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:42.426596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d3535-d505-4660-9308-d79b35c20bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:42.426864 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d3535-d505-4660-9308-d79b35c20bd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:42.427548 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d3535-d505-4660-9308-d79b35c20bd7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:42.427875 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d3535-d505-4660-9308-d79b35c20bd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:42.429802 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d3535-d505-4660-9308-d79b35c20bd7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:42.437899 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 94/192] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:42.438501 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d3535-d505-4660-9308-d79b35c20bd7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:42.469896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:42.482016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:42.515877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcfbcf5-ca92-4d55-a539-703e6559156d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:42.541588 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d3535-d505-4660-9308-d79b35c20bd7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:42.550493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eff8b3-30f3-48d0-a515-010a6c64e1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:42.551133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eff8b3-30f3-48d0-a515-010a6c64e1da None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:42.551445 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eff8b3-30f3-48d0-a515-010a6c64e1da None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:42.551719 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eff8b3-30f3-48d0-a515-010a6c64e1da None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:42.561680 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eff8b3-30f3-48d0-a515-010a6c64e1da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:42.567442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-69eff8b3-30f3-48d0-a515-010a6c64e1da None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:22:42.567442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:42.567442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:42.567442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:42.567442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:42.567442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:42.567442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:42.567442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:42.567442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:42.567442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:42.567442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:42.567442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:42.567442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:42.567442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:42.567442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:42.567442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:42.568310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:42.568310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:42.568310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:42.568310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:42.568310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:42.568310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:42.568310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:42.568310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:42.568310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:42.568310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:42.568310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:42.568310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:42.568310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:42.568310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:42.568310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:42.568310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:42.569099 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:42.569099 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:42.569099 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:42.569099 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:42.569099 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:42.569099 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:42.569099 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:42.569099 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:42.569099 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:42.569099 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:42.569099 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:22:42.569099 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:22:42.569099 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 99/193] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:42.569763 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eff8b3-30f3-48d0-a515-010a6c64e1da None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:42.578159 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96bf791-a254-4cfe-93fb-af29ef706d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:42.578772 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96bf791-a254-4cfe-93fb-af29ef706d5e None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:42.579049 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96bf791-a254-4cfe-93fb-af29ef706d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:42.579354 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96bf791-a254-4cfe-93fb-af29ef706d5e None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:42.580282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96bf791-a254-4cfe-93fb-af29ef706d5e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:42.581429 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96bf791-a254-4cfe-93fb-af29ef706d5e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:42.650527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 95/194] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 12:22:42 2022] POST /identity/v3/users => generated 297 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:22:42.651406 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96bf791-a254-4cfe-93fb-af29ef706d5e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:42.673014 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eff8b3-30f3-48d0-a515-010a6c64e1da None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:42.704736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:42.715541 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7ae0c-639e-4860-a975-f2499f47641b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:42.716606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7ae0c-639e-4860-a975-f2499f47641b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:42.716954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7ae0c-639e-4860-a975-f2499f47641b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:42.717571 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7ae0c-639e-4860-a975-f2499f47641b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:42.718488 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7ae0c-639e-4860-a975-f2499f47641b None admin] RBAC: Adding query filter params (role.id=0f58d2782a2840648bbbc2ab26ec02df, scope.project.id=9c9a0023662b41afa1d7bec05fd12eda, user.id=a43f0c8b634b4f2997cbdf44c7dd2c5a) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:42.719022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7ae0c-639e-4860-a975-f2499f47641b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:42.720971 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7ae0c-639e-4860-a975-f2499f47641b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:42.728233 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 100/195] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:22:42 2022] GET /identity/v3/role_assignments?user.id=a43f0c8b634b4f2997cbdf44c7dd2c5a&scope.project.id=9c9a0023662b41afa1d7bec05fd12eda&role.id=0f58d2782a2840648bbbc2ab26ec02df => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:42.736372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7ae0c-639e-4860-a975-f2499f47641b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:42.755505 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:42.785965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96bf791-a254-4cfe-93fb-af29ef706d5e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:42.835144 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d033b-46b0-4efd-9c26-0fb95cd8c663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:42.835924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d033b-46b0-4efd-9c26-0fb95cd8c663 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:42.836203 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d033b-46b0-4efd-9c26-0fb95cd8c663 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:42.836472 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d033b-46b0-4efd-9c26-0fb95cd8c663 None admin] PATH_INFO: `/v3/users/8fc8b8a221da411786fd37147e42f189` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:42.852123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8d033b-46b0-4efd-9c26-0fb95cd8c663 None admin] RBAC: Authorizing `identity:get_user(user_id=8fc8b8a221da411786fd37147e42f189)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:42.853652 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8d033b-46b0-4efd-9c26-0fb95cd8c663 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:42.856782 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 96/196] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:42 2022] GET /identity/v3/users/8fc8b8a221da411786fd37147e42f189 => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:42.857316 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d033b-46b0-4efd-9c26-0fb95cd8c663 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:42.873951 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7ae0c-639e-4860-a975-f2499f47641b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:42.915282 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ca127-5e60-48b9-8368-307f46b204b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:42.916160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ca127-5e60-48b9-8368-307f46b204b7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:42.916616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ca127-5e60-48b9-8368-307f46b204b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:42.916888 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ca127-5e60-48b9-8368-307f46b204b7 None admin] PATH_INFO: `/v3/users/8fc8b8a221da411786fd37147e42f189` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:42.918053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ca127-5e60-48b9-8368-307f46b204b7 None admin] RBAC: Authorizing `identity:get_user(user_id=8fc8b8a221da411786fd37147e42f189)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:42.919540 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ca127-5e60-48b9-8368-307f46b204b7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:42.921174 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 101/197] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:42 2022] GET /identity/v3/users/8fc8b8a221da411786fd37147e42f189 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:42.925384 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ca127-5e60-48b9-8368-307f46b204b7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:42.988517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d033b-46b0-4efd-9c26-0fb95cd8c663 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:42.994321 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:43.032032 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712c063-7278-453f-8cc3-984f00edeb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:43.032659 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712c063-7278-453f-8cc3-984f00edeb10 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:43.032943 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712c063-7278-453f-8cc3-984f00edeb10 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:43.033781 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712c063-7278-453f-8cc3-984f00edeb10 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:43.033909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7712c063-7278-453f-8cc3-984f00edeb10 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:43.034166 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7712c063-7278-453f-8cc3-984f00edeb10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:43.035557 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7712c063-7278-453f-8cc3-984f00edeb10 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:43.046630 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 97/198] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 12:22:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:43.047431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712c063-7278-453f-8cc3-984f00edeb10 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:43.049465 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ca127-5e60-48b9-8368-307f46b204b7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:43.051546 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:43.086944 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbcdb02-1b2a-470c-aab0-27125ced5d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:43.087639 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcdb02-1b2a-470c-aab0-27125ced5d98 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:43.087920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcdb02-1b2a-470c-aab0-27125ced5d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:43.088192 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcdb02-1b2a-470c-aab0-27125ced5d98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:43.158681 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712c063-7278-453f-8cc3-984f00edeb10 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:43.178031 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbcdb02-1b2a-470c-aab0-27125ced5d98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:43.180110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbcdb02-1b2a-470c-aab0-27125ced5d98 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:43.196251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54293df-f107-41d0-b0b7-22ac71548898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:43.196868 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54293df-f107-41d0-b0b7-22ac71548898 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:43.197219 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54293df-f107-41d0-b0b7-22ac71548898 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:43.197470 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54293df-f107-41d0-b0b7-22ac71548898 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:43.253105 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:43.288342 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54293df-f107-41d0-b0b7-22ac71548898 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:43.290986 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54293df-f107-41d0-b0b7-22ac71548898 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:43.293597 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:43.310065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 102/199] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:22:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:22:43.310065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbcdb02-1b2a-470c-aab0-27125ced5d98 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:43.408449 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 98/200] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:22:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:22:43.409435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54293df-f107-41d0-b0b7-22ac71548898 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:43.425037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbcdb02-1b2a-470c-aab0-27125ced5d98 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:43.463112 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09aa52c-b83e-4001-a17f-a13303e1121b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:43.463826 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09aa52c-b83e-4001-a17f-a13303e1121b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:43.464099 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09aa52c-b83e-4001-a17f-a13303e1121b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:43.464380 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09aa52c-b83e-4001-a17f-a13303e1121b None admin] PATH_INFO: `/v3/users/9bdcd0c1fbd34f56abe6a3fd9c128c69` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:43.465534 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09aa52c-b83e-4001-a17f-a13303e1121b None admin] RBAC: Authorizing `identity:get_user(user_id=9bdcd0c1fbd34f56abe6a3fd9c128c69)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:43.467128 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09aa52c-b83e-4001-a17f-a13303e1121b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:43.468886 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 103/201] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:43 2022] GET /identity/v3/users/9bdcd0c1fbd34f56abe6a3fd9c128c69 => generated 289 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:43.471224 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09aa52c-b83e-4001-a17f-a13303e1121b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:43.501054 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:43.518420 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54293df-f107-41d0-b0b7-22ac71548898 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:43.520331 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:43.556241 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e6a57-0b82-4f19-ad22-090c054544af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:43.557294 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e6a57-0b82-4f19-ad22-090c054544af None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:43.557799 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e6a57-0b82-4f19-ad22-090c054544af None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:43.558259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e6a57-0b82-4f19-ad22-090c054544af None admin] PATH_INFO: `/v3/projects/b287589e923744ec9ed8d9c9f4196506` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:43.559852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e6a57-0b82-4f19-ad22-090c054544af None admin] RBAC: Authorizing `identity:get_project(project_id=b287589e923744ec9ed8d9c9f4196506)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:43.561566 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e6a57-0b82-4f19-ad22-090c054544af None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:43.563937 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 99/202] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:22:43 2022] GET /identity/v3/projects/b287589e923744ec9ed8d9c9f4196506 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:43.564982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e6a57-0b82-4f19-ad22-090c054544af None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:43.578953 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09aa52c-b83e-4001-a17f-a13303e1121b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:43.614242 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8fe668-4ba5-4abb-beda-7d2e29b120d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:43.614891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8fe668-4ba5-4abb-beda-7d2e29b120d1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:43.615196 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8fe668-4ba5-4abb-beda-7d2e29b120d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:43.616098 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8fe668-4ba5-4abb-beda-7d2e29b120d1 None admin] PATH_INFO: `/v3/projects/b287589e923744ec9ed8d9c9f4196506` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:43.617466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8fe668-4ba5-4abb-beda-7d2e29b120d1 None admin] RBAC: Authorizing `identity:get_project(project_id=b287589e923744ec9ed8d9c9f4196506)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:43.619039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8fe668-4ba5-4abb-beda-7d2e29b120d1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:43.621187 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 104/203] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:22:43 2022] GET /identity/v3/projects/b287589e923744ec9ed8d9c9f4196506 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:43.622050 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8fe668-4ba5-4abb-beda-7d2e29b120d1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:43.684482 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e6a57-0b82-4f19-ad22-090c054544af None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:43.721061 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc130d0-7cbd-4b76-bde4-cae6957e585d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:43.721671 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc130d0-7cbd-4b76-bde4-cae6957e585d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:43.722061 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:43.722751 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc130d0-7cbd-4b76-bde4-cae6957e585d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:43.722994 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc130d0-7cbd-4b76-bde4-cae6957e585d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:43.729707 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8fe668-4ba5-4abb-beda-7d2e29b120d1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:43.765690 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:43.771654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a15837-2021-4eda-b522-4be292f10439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:43.772582 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a15837-2021-4eda-b522-4be292f10439 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:43.773060 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a15837-2021-4eda-b522-4be292f10439 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:43.773363 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a15837-2021-4eda-b522-4be292f10439 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:43.774453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a15837-2021-4eda-b522-4be292f10439 None admin] RBAC: Adding query filter params (role.id=0f58d2782a2840648bbbc2ab26ec02df, scope.project.id=b287589e923744ec9ed8d9c9f4196506, user.id=8fc8b8a221da411786fd37147e42f189) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:43.774788 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a15837-2021-4eda-b522-4be292f10439 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:43.776621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a15837-2021-4eda-b522-4be292f10439 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:43.783410 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 105/204] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:22:43 2022] GET /identity/v3/role_assignments?user.id=8fc8b8a221da411786fd37147e42f189&scope.project.id=b287589e923744ec9ed8d9c9f4196506&role.id=0f58d2782a2840648bbbc2ab26ec02df => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:43.784980 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a15837-2021-4eda-b522-4be292f10439 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:43.804253 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc130d0-7cbd-4b76-bde4-cae6957e585d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:43.806362 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc130d0-7cbd-4b76-bde4-cae6957e585d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:43.907749 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a15837-2021-4eda-b522-4be292f10439 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:43.946056 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5696e6-a985-4201-a715-7de06c85085d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:43.947010 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5696e6-a985-4201-a715-7de06c85085d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:43.947660 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5696e6-a985-4201-a715-7de06c85085d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:43.948115 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5696e6-a985-4201-a715-7de06c85085d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:43.949022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5696e6-a985-4201-a715-7de06c85085d None admin] RBAC: Adding query filter params (role.id=bffbe227cda1418cbb58394ddff6f861, scope.project.id=b287589e923744ec9ed8d9c9f4196506, user.id=8fc8b8a221da411786fd37147e42f189) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:43.949619 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5696e6-a985-4201-a715-7de06c85085d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:43.950250 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 100/205] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:22:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:22:43.951502 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5696e6-a985-4201-a715-7de06c85085d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:43.957027 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc130d0-7cbd-4b76-bde4-cae6957e585d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:43.960730 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 106/206] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:22:43 2022] GET /identity/v3/role_assignments?user.id=8fc8b8a221da411786fd37147e42f189&scope.project.id=b287589e923744ec9ed8d9c9f4196506&role.id=bffbe227cda1418cbb58394ddff6f861 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:44.057963 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:44.092047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc130d0-7cbd-4b76-bde4-cae6957e585d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:44.145595 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81921e7-af73-4660-a67d-7f20f9241af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:44.146251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81921e7-af73-4660-a67d-7f20f9241af4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:44.146525 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81921e7-af73-4660-a67d-7f20f9241af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:44.147369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81921e7-af73-4660-a67d-7f20f9241af4 None admin] PATH_INFO: `/v3/projects/9c9a0023662b41afa1d7bec05fd12eda` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:44.152012 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81921e7-af73-4660-a67d-7f20f9241af4 None admin] RBAC: Authorizing `identity:get_project(project_id=9c9a0023662b41afa1d7bec05fd12eda)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:44.153747 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81921e7-af73-4660-a67d-7f20f9241af4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:44.155742 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 101/207] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:22:43 2022] GET /identity/v3/projects/9c9a0023662b41afa1d7bec05fd12eda => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:44.163035 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5696e6-a985-4201-a715-7de06c85085d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:44.206267 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:44.317808 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5696e6-a985-4201-a715-7de06c85085d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:44.364931 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d968544-133c-4397-b35d-b10f07cec9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:44.366301 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d968544-133c-4397-b35d-b10f07cec9b0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:44.366676 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d968544-133c-4397-b35d-b10f07cec9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:44.366955 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d968544-133c-4397-b35d-b10f07cec9b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:44.367720 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d968544-133c-4397-b35d-b10f07cec9b0 None admin] RBAC: Adding query filter params (role.id=78c5ea5c9f154c6da4912cfc338b6474, scope.project.id=9c9a0023662b41afa1d7bec05fd12eda, user.id=9bdcd0c1fbd34f56abe6a3fd9c128c69) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:44.368030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d968544-133c-4397-b35d-b10f07cec9b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:44.369669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d968544-133c-4397-b35d-b10f07cec9b0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:44.378378 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 107/208] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:22:44 2022] GET /identity/v3/role_assignments?user.id=9bdcd0c1fbd34f56abe6a3fd9c128c69&scope.project.id=9c9a0023662b41afa1d7bec05fd12eda&role.id=78c5ea5c9f154c6da4912cfc338b6474 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:45.128920 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1237d-2183-4553-94f8-7f7a7b1e887b None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:45.129427 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1237d-2183-4553-94f8-7f7a7b1e887b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:45.129500 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1237d-2183-4553-94f8-7f7a7b1e887b None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:45.131598 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 102/209] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:45.137074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c361f-e106-414f-9cce-0d8627afbf25 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:45.137513 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c361f-e106-414f-9cce-0d8627afbf25 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:45.137684 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c361f-e106-414f-9cce-0d8627afbf25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:45.157283 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-132c361f-e106-414f-9cce-0d8627afbf25 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:45.158511 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-132c361f-e106-414f-9cce-0d8627afbf25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:45.168326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49874ae-5b6b-4182-a337-599c858dddcd None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:45.168653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49874ae-5b6b-4182-a337-599c858dddcd None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:45.168878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49874ae-5b6b-4182-a337-599c858dddcd None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:45.170209 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 103/210] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:45.175104 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804006a8-bdf5-452e-8a59-af319a7b813f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:45.175421 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804006a8-bdf5-452e-8a59-af319a7b813f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:45.175690 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804006a8-bdf5-452e-8a59-af319a7b813f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:45.195864 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-804006a8-bdf5-452e-8a59-af319a7b813f None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:45.197115 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-804006a8-bdf5-452e-8a59-af319a7b813f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:45.236261 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 108/211] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:45.265200 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c361f-e106-414f-9cce-0d8627afbf25 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:45.276544 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 104/212] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:45.289077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:45.300018 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804006a8-bdf5-452e-8a59-af319a7b813f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:45.373810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:45.383315 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c361f-e106-414f-9cce-0d8627afbf25 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:45.410045 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804006a8-bdf5-452e-8a59-af319a7b813f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:45.422499 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c850d1-e2b8-4839-95d7-40674d4a97d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:45.423339 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c850d1-e2b8-4839-95d7-40674d4a97d1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:45.423716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c850d1-e2b8-4839-95d7-40674d4a97d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:45.424075 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c850d1-e2b8-4839-95d7-40674d4a97d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:45.436099 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c850d1-e2b8-4839-95d7-40674d4a97d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:45.442940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b9c850d1-e2b8-4839-95d7-40674d4a97d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:22:45.442940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:45.442940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:45.442940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:45.442940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:45.442940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:45.442940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:45.442940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:45.442940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:45.442940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:45.442940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:45.442940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:45.442940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:45.442940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:45.442940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:45.442940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:45.443998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:45.443998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:45.443998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:45.443998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:45.443998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:45.443998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:45.443998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:45.443998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:45.443998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:45.443998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:45.443998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:45.443998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:45.443998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:45.443998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:45.443998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:45.443998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:45.444787 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:45.444787 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:45.444787 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:45.444787 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:45.444787 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:45.444787 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:45.444787 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:45.444787 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:45.444787 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:45.444787 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:45.444787 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:22:45.444787 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:22:45.445396 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 109/213] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:22:45.450810 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c850d1-e2b8-4839-95d7-40674d4a97d1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:45.453443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5369ab79-91dd-4fcb-bd9d-d5181b618f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:45.454452 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369ab79-91dd-4fcb-bd9d-d5181b618f79 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:45.455053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369ab79-91dd-4fcb-bd9d-d5181b618f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:45.455461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369ab79-91dd-4fcb-bd9d-d5181b618f79 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:45.456655 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5369ab79-91dd-4fcb-bd9d-d5181b618f79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:45.458154 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5369ab79-91dd-4fcb-bd9d-d5181b618f79 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:45.464899 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 105/214] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:22:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:45.465837 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5369ab79-91dd-4fcb-bd9d-d5181b618f79 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:45.542674 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:45.566421 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c850d1-e2b8-4839-95d7-40674d4a97d1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:45.580234 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5369ab79-91dd-4fcb-bd9d-d5181b618f79 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:45.602893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96bf0a-f80e-475c-96dd-a3cf3ec9cd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:45.603508 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96bf0a-f80e-475c-96dd-a3cf3ec9cd41 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:45.603787 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96bf0a-f80e-475c-96dd-a3cf3ec9cd41 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:45.604055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96bf0a-f80e-475c-96dd-a3cf3ec9cd41 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:45.604710 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c96bf0a-f80e-475c-96dd-a3cf3ec9cd41 None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:45.605068 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c96bf0a-f80e-475c-96dd-a3cf3ec9cd41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:45.606385 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c96bf0a-f80e-475c-96dd-a3cf3ec9cd41 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:45.615210 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 110/215] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 12:22:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:45.621174 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96bf0a-f80e-475c-96dd-a3cf3ec9cd41 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:45.624005 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b1c3d-bcc3-42e8-985d-25eb2e48a3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:45.625661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b1c3d-bcc3-42e8-985d-25eb2e48a3b9 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:45.626127 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b1c3d-bcc3-42e8-985d-25eb2e48a3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:45.626573 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b1c3d-bcc3-42e8-985d-25eb2e48a3b9 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:45.627424 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b1c3d-bcc3-42e8-985d-25eb2e48a3b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:45.629017 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b1c3d-bcc3-42e8-985d-25eb2e48a3b9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:45.642016 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:45.731004 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96bf0a-f80e-475c-96dd-a3cf3ec9cd41 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:45.740137 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 106/216] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 12:22:45 2022] POST /identity/v3/users => generated 297 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:22:45.779147 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf16518e-d52b-4c46-81da-8099f2047a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:45.780077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16518e-d52b-4c46-81da-8099f2047a31 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:45.780607 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16518e-d52b-4c46-81da-8099f2047a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:45.781150 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16518e-d52b-4c46-81da-8099f2047a31 None admin] PATH_INFO: `/v3/users/a54e7225e12b498b9af200aa54a186ad` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:45.793129 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:45.800597 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf16518e-d52b-4c46-81da-8099f2047a31 None admin] RBAC: Authorizing `identity:get_user(user_id=a54e7225e12b498b9af200aa54a186ad)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:45.802141 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf16518e-d52b-4c46-81da-8099f2047a31 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:45.805100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 111/217] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:45 2022] GET /identity/v3/users/a54e7225e12b498b9af200aa54a186ad => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:45.812164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b1c3d-bcc3-42e8-985d-25eb2e48a3b9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:45.927348 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b1c3d-bcc3-42e8-985d-25eb2e48a3b9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:45.965716 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df7364-591e-4e56-b41b-ee6961c9af2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:45.966343 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df7364-591e-4e56-b41b-ee6961c9af2c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:45.966624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df7364-591e-4e56-b41b-ee6961c9af2c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:45.966898 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df7364-591e-4e56-b41b-ee6961c9af2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:46.031448 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:46.043351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df7364-591e-4e56-b41b-ee6961c9af2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:46.044896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df7364-591e-4e56-b41b-ee6961c9af2c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:46.104944 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfe349-9193-4648-b5fb-1d2db899b307 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:46.105258 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfe349-9193-4648-b5fb-1d2db899b307 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:46.105477 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfe349-9193-4648-b5fb-1d2db899b307 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:46.106964 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 112/218] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:46.113617 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a8dbd-f4eb-4527-a75f-03f3600ab89a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:46.114057 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a8dbd-f4eb-4527-a75f-03f3600ab89a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:46.114651 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a8dbd-f4eb-4527-a75f-03f3600ab89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:46.138544 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-280a8dbd-f4eb-4527-a75f-03f3600ab89a None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:46.140433 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-280a8dbd-f4eb-4527-a75f-03f3600ab89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:46.159491 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 107/219] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:22:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:22:46.166506 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df7364-591e-4e56-b41b-ee6961c9af2c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:46.225792 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 113/220] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:22:46.251771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a8dbd-f4eb-4527-a75f-03f3600ab89a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:46.282960 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:46.287831 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df7364-591e-4e56-b41b-ee6961c9af2c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:46.327913 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d415b47c-bf53-4c81-9fd1-1a9dcede31ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:46.328560 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415b47c-bf53-4c81-9fd1-1a9dcede31ee None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:46.328842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415b47c-bf53-4c81-9fd1-1a9dcede31ee None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:46.329385 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415b47c-bf53-4c81-9fd1-1a9dcede31ee None admin] PATH_INFO: `/v3/projects/b287589e923744ec9ed8d9c9f4196506` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:46.330554 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d415b47c-bf53-4c81-9fd1-1a9dcede31ee None admin] RBAC: Authorizing `identity:get_project(project_id=b287589e923744ec9ed8d9c9f4196506)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:46.332047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d415b47c-bf53-4c81-9fd1-1a9dcede31ee None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:46.334069 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 108/221] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:22:46 2022] GET /identity/v3/projects/b287589e923744ec9ed8d9c9f4196506 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:46.346778 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d415b47c-bf53-4c81-9fd1-1a9dcede31ee None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:46.414888 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a8dbd-f4eb-4527-a75f-03f3600ab89a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:46.451591 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:46.454883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc79bc01-223b-43f8-ad85-43f9c56bd3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:46.455926 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79bc01-223b-43f8-ad85-43f9c56bd3e5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:46.456724 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79bc01-223b-43f8-ad85-43f9c56bd3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:46.457197 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79bc01-223b-43f8-ad85-43f9c56bd3e5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:46.472655 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc79bc01-223b-43f8-ad85-43f9c56bd3e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:46.475420 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d415b47c-bf53-4c81-9fd1-1a9dcede31ee None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:46.480282 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-bc79bc01-223b-43f8-ad85-43f9c56bd3e5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:22:46.480282 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:46.480282 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:46.480282 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:46.480282 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:46.480282 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:46.480282 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:46.480282 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:46.480282 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:46.480282 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:46.480282 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:46.480282 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:46.480282 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:46.480282 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:46.480282 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:46.480282 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:46.481573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:46.481573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:46.481573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:46.481573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:46.481573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:46.481573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:46.481573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:46.481573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:46.481573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:46.481573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:46.481573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:46.481573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:46.481573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:46.481573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:46.481573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:46.481573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:46.482586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:46.482586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:46.482586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:46.482586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:46.482586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:46.482586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:46.482586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:46.482586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:46.482586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:46.482586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:46.482586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:22:46.482586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:22:46.483360 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 114/222] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 12:22:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:46.485722 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4aa5d-dbcc-4a17-8c53-f54e2df9c9d5 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:46.486015 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4aa5d-dbcc-4a17-8c53-f54e2df9c9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:46.486291 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4aa5d-dbcc-4a17-8c53-f54e2df9c9d5 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:46.488047 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 115/223] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:46.493851 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4aa5d-dbcc-4a17-8c53-f54e2df9c9d5 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:46.530986 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fb6be-5adf-4333-b75f-7451864b3b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:46.531734 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fb6be-5adf-4333-b75f-7451864b3b4b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:46.532020 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fb6be-5adf-4333-b75f-7451864b3b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:46.532293 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fb6be-5adf-4333-b75f-7451864b3b4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:46.533039 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fb6be-5adf-4333-b75f-7451864b3b4b None admin] RBAC: Adding query filter params (role.id=872ba0c4212a4684b539a68fc2c20b25, scope.project.id=b287589e923744ec9ed8d9c9f4196506, user.id=a54e7225e12b498b9af200aa54a186ad) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:46.533348 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fb6be-5adf-4333-b75f-7451864b3b4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:46.534791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fb6be-5adf-4333-b75f-7451864b3b4b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:46.541817 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 109/224] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:22:46 2022] GET /identity/v3/role_assignments?user.id=a54e7225e12b498b9af200aa54a186ad&scope.project.id=b287589e923744ec9ed8d9c9f4196506&role.id=872ba0c4212a4684b539a68fc2c20b25 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:46.547302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4100ae-4833-4dc8-9df7-992d0310005c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:46.547302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4100ae-4833-4dc8-9df7-992d0310005c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:46.547302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4100ae-4833-4dc8-9df7-992d0310005c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:46.578228 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-1f4100ae-4833-4dc8-9df7-992d0310005c None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:46.579475 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1f4100ae-4833-4dc8-9df7-992d0310005c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:46.626566 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:46.631066 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4aa5d-dbcc-4a17-8c53-f54e2df9c9d5 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:46.684938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea42b4-f978-4e7b-8636-21ebea1f0830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:46.685580 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea42b4-f978-4e7b-8636-21ebea1f0830 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:46.685859 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea42b4-f978-4e7b-8636-21ebea1f0830 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:46.686130 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea42b4-f978-4e7b-8636-21ebea1f0830 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:46.686809 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea42b4-f978-4e7b-8636-21ebea1f0830 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:46.687115 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea42b4-f978-4e7b-8636-21ebea1f0830 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:46.688573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea42b4-f978-4e7b-8636-21ebea1f0830 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:46.711532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 116/225] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:22:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:46.717914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea42b4-f978-4e7b-8636-21ebea1f0830 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:46.734753 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 110/226] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:46.759117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4100ae-4833-4dc8-9df7-992d0310005c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:46.808718 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:46.915548 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4100ae-4833-4dc8-9df7-992d0310005c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:46.921307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea42b4-f978-4e7b-8636-21ebea1f0830 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:46.966854 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31057f-cc22-43b2-bec5-133c5b111ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:46.967905 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31057f-cc22-43b2-bec5-133c5b111ddb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:46.968389 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31057f-cc22-43b2-bec5-133c5b111ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:46.968860 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31057f-cc22-43b2-bec5-133c5b111ddb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:46.989654 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31057f-cc22-43b2-bec5-133c5b111ddb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:46.996222 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92884b-547f-49ec-943d-dc295845b58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:46.997160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92884b-547f-49ec-943d-dc295845b58d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:46.997660 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92884b-547f-49ec-943d-dc295845b58d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:46.998133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92884b-547f-49ec-943d-dc295845b58d None admin] PATH_INFO: `/v3/users/8fc8b8a221da411786fd37147e42f189` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:47.000144 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-8d31057f-cc22-43b2-bec5-133c5b111ddb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:22:47.000144 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:47.000144 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:47.000144 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:47.000144 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:47.000144 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:47.000144 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:47.000144 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:47.000144 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:47.000144 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:47.000144 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:47.000144 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:47.000144 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:47.000144 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:47.000144 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:47.000144 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:47.010970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:47.010970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:47.010970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:47.010970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:47.010970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:47.010970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:47.010970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:47.010970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:47.010970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:47.010970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:47.010970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:47.010970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:47.010970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:47.010970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:47.010970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:47.010970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:47.011806 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:47.011806 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:47.011806 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:47.011806 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:47.011806 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:47.011806 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:47.011806 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:47.011806 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:47.011806 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:47.011806 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:47.011806 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:22:47.011806 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:22:47.012394 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a92884b-547f-49ec-943d-dc295845b58d None admin] RBAC: Authorizing `identity:get_user(user_id=8fc8b8a221da411786fd37147e42f189)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:47.012394 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a92884b-547f-49ec-943d-dc295845b58d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:47.012394 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 117/227] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:46 2022] GET /identity/v3/users/8fc8b8a221da411786fd37147e42f189 => generated 283 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:47.012394 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92884b-547f-49ec-943d-dc295845b58d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:47.013265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 111/228] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:22:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:22:47.016898 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31057f-cc22-43b2-bec5-133c5b111ddb None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:47.046604 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:47.168980 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31057f-cc22-43b2-bec5-133c5b111ddb None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:47.210112 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92884b-547f-49ec-943d-dc295845b58d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:47.214439 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54183500-3d87-453b-8ffb-199bcff3ad08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:47.215063 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54183500-3d87-453b-8ffb-199bcff3ad08 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:47.215375 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54183500-3d87-453b-8ffb-199bcff3ad08 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:47.215647 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54183500-3d87-453b-8ffb-199bcff3ad08 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:47.216329 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54183500-3d87-453b-8ffb-199bcff3ad08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:47.216750 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54183500-3d87-453b-8ffb-199bcff3ad08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:47.218246 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54183500-3d87-453b-8ffb-199bcff3ad08 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:47.231590 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:47.236240 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 112/229] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:47.244780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54183500-3d87-453b-8ffb-199bcff3ad08 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:47.267347 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51f3b8-1604-4d5c-81c7-d2c526937927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:47.268270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51f3b8-1604-4d5c-81c7-d2c526937927 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:47.268936 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51f3b8-1604-4d5c-81c7-d2c526937927 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:47.269430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51f3b8-1604-4d5c-81c7-d2c526937927 None admin] PATH_INFO: `/v3/projects/b287589e923744ec9ed8d9c9f4196506` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:47.270730 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a51f3b8-1604-4d5c-81c7-d2c526937927 None admin] RBAC: Authorizing `identity:get_project(project_id=b287589e923744ec9ed8d9c9f4196506)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:47.273507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a51f3b8-1604-4d5c-81c7-d2c526937927 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:47.275780 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 118/230] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:22:47 2022] GET /identity/v3/projects/b287589e923744ec9ed8d9c9f4196506 => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:47.283994 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51f3b8-1604-4d5c-81c7-d2c526937927 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:47.392150 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:47.435056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54183500-3d87-453b-8ffb-199bcff3ad08 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:47.471849 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51f3b8-1604-4d5c-81c7-d2c526937927 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:47.496714 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e79360-2892-453c-8699-147f799ea63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:47.497278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e79360-2892-453c-8699-147f799ea63a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:47.497554 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e79360-2892-453c-8699-147f799ea63a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:47.497823 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e79360-2892-453c-8699-147f799ea63a None admin] PATH_INFO: `/v3/users/8fc8b8a221da411786fd37147e42f189` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:47.499799 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e79360-2892-453c-8699-147f799ea63a None admin] RBAC: Authorizing `identity:get_user(user_id=8fc8b8a221da411786fd37147e42f189)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:47.501227 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e79360-2892-453c-8699-147f799ea63a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:47.502798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 113/231] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:47 2022] GET /identity/v3/users/8fc8b8a221da411786fd37147e42f189 => generated 283 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:47.509724 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e79360-2892-453c-8699-147f799ea63a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:47.533539 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824eaad-4e41-405c-aabf-825c529e63e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:47.534197 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824eaad-4e41-405c-aabf-825c529e63e8 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:47.534473 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824eaad-4e41-405c-aabf-825c529e63e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:47.534744 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824eaad-4e41-405c-aabf-825c529e63e8 None admin] PATH_INFO: `/v3/projects/b287589e923744ec9ed8d9c9f4196506/users/8fc8b8a221da411786fd37147e42f189/roles/0f58d2782a2840648bbbc2ab26ec02df` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:47.540499 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5824eaad-4e41-405c-aabf-825c529e63e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=b287589e923744ec9ed8d9c9f4196506, user_id=8fc8b8a221da411786fd37147e42f189, role_id=0f58d2782a2840648bbbc2ab26ec02df)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:47.541965 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5824eaad-4e41-405c-aabf-825c529e63e8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:47.633288 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e79360-2892-453c-8699-147f799ea63a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:47.653040 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 119/232] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 12:22:47 2022] PUT /identity/v3/projects/b287589e923744ec9ed8d9c9f4196506/users/8fc8b8a221da411786fd37147e42f189/roles/0f58d2782a2840648bbbc2ab26ec02df => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:22:47.681445 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bda6af-3702-4be0-9870-016179d05686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:47.682085 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bda6af-3702-4be0-9870-016179d05686 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:47.682364 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bda6af-3702-4be0-9870-016179d05686 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:47.682633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bda6af-3702-4be0-9870-016179d05686 None admin] PATH_INFO: `/v3/projects/b287589e923744ec9ed8d9c9f4196506` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:47.683781 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bda6af-3702-4be0-9870-016179d05686 None admin] RBAC: Authorizing `identity:get_project(project_id=b287589e923744ec9ed8d9c9f4196506)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:47.685206 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bda6af-3702-4be0-9870-016179d05686 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:47.691621 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 114/233] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:22:47 2022] GET /identity/v3/projects/b287589e923744ec9ed8d9c9f4196506 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:47.693264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824eaad-4e41-405c-aabf-825c529e63e8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:47.752817 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:47.813876 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824eaad-4e41-405c-aabf-825c529e63e8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:47.851529 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f29d707-2992-4b36-b67f-748e2dee315a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:47.852209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29d707-2992-4b36-b67f-748e2dee315a None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:47.852485 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29d707-2992-4b36-b67f-748e2dee315a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:47.852758 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29d707-2992-4b36-b67f-748e2dee315a None admin] PATH_INFO: `/v3/projects/b287589e923744ec9ed8d9c9f4196506/users/8fc8b8a221da411786fd37147e42f189/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:47.854841 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f29d707-2992-4b36-b67f-748e2dee315a None admin] RBAC: Authorizing `identity:create_grant(project_id=b287589e923744ec9ed8d9c9f4196506, user_id=8fc8b8a221da411786fd37147e42f189, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:47.856380 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f29d707-2992-4b36-b67f-748e2dee315a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:47.964713 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 120/234] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 12:22:47 2022] PUT /identity/v3/projects/b287589e923744ec9ed8d9c9f4196506/users/8fc8b8a221da411786fd37147e42f189/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:22:48.403860 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005e074-325e-446f-8fb4-a51c8c477734 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:48.404358 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005e074-325e-446f-8fb4-a51c8c477734 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:48.404430 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005e074-325e-446f-8fb4-a51c8c477734 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:48.406011 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 115/235] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:48.410951 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d834d-3f2d-4af9-92d5-f3c0b9615ae1 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:48.411422 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d834d-3f2d-4af9-92d5-f3c0b9615ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:48.411605 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d834d-3f2d-4af9-92d5-f3c0b9615ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:48.439262 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-402d834d-3f2d-4af9-92d5-f3c0b9615ae1 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:48.440425 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-402d834d-3f2d-4af9-92d5-f3c0b9615ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:48.465738 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d42fc6-47f8-452a-9a25-b9605094e9ad None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:48.466025 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d42fc6-47f8-452a-9a25-b9605094e9ad None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:48.466310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d42fc6-47f8-452a-9a25-b9605094e9ad None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:48.467944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 116/236] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:48.473043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aad4fd-5994-4305-8e16-84162a771e1f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:48.473333 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aad4fd-5994-4305-8e16-84162a771e1f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:48.473607 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aad4fd-5994-4305-8e16-84162a771e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:48.519595 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c4aad4fd-5994-4305-8e16-84162a771e1f None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:48.520998 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c4aad4fd-5994-4305-8e16-84162a771e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:48.539618 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:48.555334 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 121/237] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:48.590692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d834d-3f2d-4af9-92d5-f3c0b9615ae1 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:48.608733 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 117/238] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:22:48.639677 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:48.655159 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aad4fd-5994-4305-8e16-84162a771e1f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:48.752038 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d834d-3f2d-4af9-92d5-f3c0b9615ae1 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:48.815746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c2662-72d2-455b-9a95-253b47f610ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:48.817866 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c2662-72d2-455b-9a95-253b47f610ca None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:48.818292 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c2662-72d2-455b-9a95-253b47f610ca None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:48.818921 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c2662-72d2-455b-9a95-253b47f610ca None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:48.840609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c2662-72d2-455b-9a95-253b47f610ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:48.848495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-e83c2662-72d2-455b-9a95-253b47f610ca None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:22:48.848495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:48.848495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:48.848495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:48.848495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:48.848495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:48.848495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:48.848495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:48.848495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:48.848495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:48.848495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:48.848495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:48.848495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:48.848495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:48.848495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:48.848495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:48.849549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:48.849549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:48.849549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:48.849549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:48.849549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:48.849549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:48.849549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:48.849549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:48.849549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:48.849549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:48.849549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:48.849549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:48.849549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:48.849549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:48.849549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:48.849549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:48.850943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:48.850943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:48.850943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:48.850943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:48.850943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:48.850943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:48.850943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:48.850943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:48.850943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:48.850943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:48.850943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:22:48.850943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:22:48.852472 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 122/239] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:22:48.858819 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c2662-72d2-455b-9a95-253b47f610ca None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:48.866984 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aad4fd-5994-4305-8e16-84162a771e1f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:48.922357 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b44c33-267d-4c1a-9d7f-7e158d26ea85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:48.922999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b44c33-267d-4c1a-9d7f-7e158d26ea85 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:48.923884 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b44c33-267d-4c1a-9d7f-7e158d26ea85 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:48.924168 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b44c33-267d-4c1a-9d7f-7e158d26ea85 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:48.925292 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b44c33-267d-4c1a-9d7f-7e158d26ea85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:48.927200 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b44c33-267d-4c1a-9d7f-7e158d26ea85 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:48.930141 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 118/240] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:22:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:48.935480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355f51c-a2c0-4e08-a0aa-e9130f6aa7fc None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:48.935794 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355f51c-a2c0-4e08-a0aa-e9130f6aa7fc None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:48.936077 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355f51c-a2c0-4e08-a0aa-e9130f6aa7fc None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:48.937491 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 119/241] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:48.938782 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355f51c-a2c0-4e08-a0aa-e9130f6aa7fc None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:48.954761 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:49.007154 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:49.011817 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c2662-72d2-455b-9a95-253b47f610ca None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:49.055975 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3a824-7923-4e58-a20c-271b5f405070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:49.056595 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3a824-7923-4e58-a20c-271b5f405070 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:49.056879 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3a824-7923-4e58-a20c-271b5f405070 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:49.057154 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3a824-7923-4e58-a20c-271b5f405070 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:49.057924 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3a824-7923-4e58-a20c-271b5f405070 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:49.058237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3a824-7923-4e58-a20c-271b5f405070 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:49.059767 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3a824-7923-4e58-a20c-271b5f405070 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:49.062678 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355f51c-a2c0-4e08-a0aa-e9130f6aa7fc None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:49.074113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 123/242] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 12:22:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:49.075541 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa8488-9381-4ceb-a03e-bccc7edf603c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:49.075831 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa8488-9381-4ceb-a03e-bccc7edf603c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:49.076128 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa8488-9381-4ceb-a03e-bccc7edf603c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:49.096034 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-67aa8488-9381-4ceb-a03e-bccc7edf603c None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:49.097438 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-67aa8488-9381-4ceb-a03e-bccc7edf603c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:49.103982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e17b8f-d3e0-47a1-b591-76284619fb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:49.104602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e17b8f-d3e0-47a1-b591-76284619fb84 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:49.104880 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e17b8f-d3e0-47a1-b591-76284619fb84 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:49.105153 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e17b8f-d3e0-47a1-b591-76284619fb84 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:49.105808 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e17b8f-d3e0-47a1-b591-76284619fb84 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:49.107804 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e17b8f-d3e0-47a1-b591-76284619fb84 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:49.170860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 124/243] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:49.171843 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa8488-9381-4ceb-a03e-bccc7edf603c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:49.233096 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 120/244] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 12:22:48 2022] POST /identity/v3/users => generated 293 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:22:49.234166 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e17b8f-d3e0-47a1-b591-76284619fb84 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:49.278213 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:49.296117 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa8488-9381-4ceb-a03e-bccc7edf603c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:49.328917 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:49.333961 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699dd4d-2362-4e3d-87e2-a9e69cbf9174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:49.335464 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699dd4d-2362-4e3d-87e2-a9e69cbf9174 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:49.336583 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699dd4d-2362-4e3d-87e2-a9e69cbf9174 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:49.336583 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699dd4d-2362-4e3d-87e2-a9e69cbf9174 None admin] PATH_INFO: `/v3/users/67d9ac49973a425686988215f938c364` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:49.352609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d699dd4d-2362-4e3d-87e2-a9e69cbf9174 None admin] RBAC: Authorizing `identity:get_user(user_id=67d9ac49973a425686988215f938c364)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:49.354409 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d699dd4d-2362-4e3d-87e2-a9e69cbf9174 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:49.356407 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 125/245] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:49 2022] GET /identity/v3/users/67d9ac49973a425686988215f938c364 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:49.368251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699dd4d-2362-4e3d-87e2-a9e69cbf9174 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:49.377351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e17b8f-d3e0-47a1-b591-76284619fb84 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:49.417094 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60677e55-3177-4165-8440-c6a9679d466f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:49.418499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60677e55-3177-4165-8440-c6a9679d466f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:49.418928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60677e55-3177-4165-8440-c6a9679d466f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:49.419369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60677e55-3177-4165-8440-c6a9679d466f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:49.431566 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60677e55-3177-4165-8440-c6a9679d466f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:49.437315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-60677e55-3177-4165-8440-c6a9679d466f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:22:49.437315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:49.437315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:49.437315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:49.437315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:49.437315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:49.437315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:49.437315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:49.437315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:49.437315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:49.437315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:49.437315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:49.437315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:49.437315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:49.437315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:49.437315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:49.438210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:49.438210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:49.438210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:49.438210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:49.438210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:49.438210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:49.438210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:49.438210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:49.438210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:49.438210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:49.438210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:49.438210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:49.438210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:49.438210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:49.438210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:49.438210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:49.439117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:49.439117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:49.439117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:49.439117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:49.439117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:49.439117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:49.439117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:49.439117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:49.439117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:49.439117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:49.439117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:22:49.439117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:22:49.441638 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 121/246] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:49.449556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60677e55-3177-4165-8440-c6a9679d466f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:49.528982 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699dd4d-2362-4e3d-87e2-a9e69cbf9174 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:49.579411 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca858dd-6d7f-4785-987d-3a1186a2d552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:49.580384 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca858dd-6d7f-4785-987d-3a1186a2d552 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:49.580813 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca858dd-6d7f-4785-987d-3a1186a2d552 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:49.581394 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca858dd-6d7f-4785-987d-3a1186a2d552 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:49.600745 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:49.629652 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60677e55-3177-4165-8440-c6a9679d466f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:49.667350 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:49.680050 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca858dd-6d7f-4785-987d-3a1186a2d552 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:49.681471 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f2c55-13b6-4b62-9bce-8077a074230d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:49.682073 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca858dd-6d7f-4785-987d-3a1186a2d552 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:49.682838 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f2c55-13b6-4b62-9bce-8077a074230d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:49.682838 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f2c55-13b6-4b62-9bce-8077a074230d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:49.684557 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f2c55-13b6-4b62-9bce-8077a074230d None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:49.685255 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f2c55-13b6-4b62-9bce-8077a074230d None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:49.685631 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f2c55-13b6-4b62-9bce-8077a074230d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:49.687063 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f2c55-13b6-4b62-9bce-8077a074230d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:49.697635 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 122/247] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 12:22:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:49.703204 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f2c55-13b6-4b62-9bce-8077a074230d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:49.814973 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 126/248] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:22:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:22:49.821818 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca858dd-6d7f-4785-987d-3a1186a2d552 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:49.848116 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f2c55-13b6-4b62-9bce-8077a074230d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:49.880549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:49.898503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f827a3fa-ff1a-4c7c-82f6-b07ee1168844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:49.899226 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827a3fa-ff1a-4c7c-82f6-b07ee1168844 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:49.899511 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827a3fa-ff1a-4c7c-82f6-b07ee1168844 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:49.899784 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827a3fa-ff1a-4c7c-82f6-b07ee1168844 None admin] PATH_INFO: `/v3/users/a54e7225e12b498b9af200aa54a186ad` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:49.901122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f827a3fa-ff1a-4c7c-82f6-b07ee1168844 None admin] RBAC: Authorizing `identity:get_user(user_id=a54e7225e12b498b9af200aa54a186ad)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:49.903301 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f827a3fa-ff1a-4c7c-82f6-b07ee1168844 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:49.905807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 123/249] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:49 2022] GET /identity/v3/users/a54e7225e12b498b9af200aa54a186ad => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:49.911266 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f827a3fa-ff1a-4c7c-82f6-b07ee1168844 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:49.969432 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca858dd-6d7f-4785-987d-3a1186a2d552 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:50.003626 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:50.020746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67480ed8-2fda-403f-8df0-ed7690d5376e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:50.021399 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67480ed8-2fda-403f-8df0-ed7690d5376e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:50.023297 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67480ed8-2fda-403f-8df0-ed7690d5376e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:50.023297 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67480ed8-2fda-403f-8df0-ed7690d5376e None admin] PATH_INFO: `/v3/projects/b287589e923744ec9ed8d9c9f4196506` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:50.024816 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67480ed8-2fda-403f-8df0-ed7690d5376e None admin] RBAC: Authorizing `identity:get_project(project_id=b287589e923744ec9ed8d9c9f4196506)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:50.026394 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67480ed8-2fda-403f-8df0-ed7690d5376e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:50.028149 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 127/250] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:22:49 2022] GET /identity/v3/projects/b287589e923744ec9ed8d9c9f4196506 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:50.033992 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67480ed8-2fda-403f-8df0-ed7690d5376e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:50.060691 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f827a3fa-ff1a-4c7c-82f6-b07ee1168844 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:50.098519 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec7c0e-13dc-4a07-a587-25c779a9d23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:50.099530 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec7c0e-13dc-4a07-a587-25c779a9d23e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:50.100068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec7c0e-13dc-4a07-a587-25c779a9d23e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:50.100538 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec7c0e-13dc-4a07-a587-25c779a9d23e None admin] PATH_INFO: `/v3/projects/b287589e923744ec9ed8d9c9f4196506` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:50.101999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec7c0e-13dc-4a07-a587-25c779a9d23e None admin] RBAC: Authorizing `identity:get_project(project_id=b287589e923744ec9ed8d9c9f4196506)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:50.104357 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec7c0e-13dc-4a07-a587-25c779a9d23e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:50.106271 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 124/251] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:22:49 2022] GET /identity/v3/projects/b287589e923744ec9ed8d9c9f4196506 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:50.107842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53995b-e6da-4782-be75-7359fb9f87f2 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:50.108368 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53995b-e6da-4782-be75-7359fb9f87f2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:50.108846 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53995b-e6da-4782-be75-7359fb9f87f2 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:50.110607 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 125/252] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:50.116589 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826f9dc-b14d-4eef-b14f-b2dcb6e8ac81 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:50.117203 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826f9dc-b14d-4eef-b14f-b2dcb6e8ac81 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:50.117677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826f9dc-b14d-4eef-b14f-b2dcb6e8ac81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:50.138519 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-2826f9dc-b14d-4eef-b14f-b2dcb6e8ac81 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:50.140244 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2826f9dc-b14d-4eef-b14f-b2dcb6e8ac81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:50.151949 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67480ed8-2fda-403f-8df0-ed7690d5376e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:50.165965 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:50.188865 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e2ccd-c422-4694-9701-d385a43048b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:50.189529 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e2ccd-c422-4694-9701-d385a43048b8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:50.189861 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e2ccd-c422-4694-9701-d385a43048b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:50.190133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e2ccd-c422-4694-9701-d385a43048b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:50.190869 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e2ccd-c422-4694-9701-d385a43048b8 None admin] RBAC: Adding query filter params (role.id=78c5ea5c9f154c6da4912cfc338b6474, scope.project.id=b287589e923744ec9ed8d9c9f4196506, user.id=67d9ac49973a425686988215f938c364) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:50.191200 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e2ccd-c422-4694-9701-d385a43048b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:50.192660 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e2ccd-c422-4694-9701-d385a43048b8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:50.205596 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 128/253] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:22:50 2022] GET /identity/v3/role_assignments?user.id=67d9ac49973a425686988215f938c364&scope.project.id=b287589e923744ec9ed8d9c9f4196506&role.id=78c5ea5c9f154c6da4912cfc338b6474 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:50.206218 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e2ccd-c422-4694-9701-d385a43048b8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:50.221260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 126/254] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:22:50.272826 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826f9dc-b14d-4eef-b14f-b2dcb6e8ac81 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:50.351328 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e2ccd-c422-4694-9701-d385a43048b8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:50.382570 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:50.418451 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84ab4f-dfd7-4be2-932d-1dd03a0eceb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:50.419605 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84ab4f-dfd7-4be2-932d-1dd03a0eceb3 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:50.421484 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84ab4f-dfd7-4be2-932d-1dd03a0eceb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:50.421907 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84ab4f-dfd7-4be2-932d-1dd03a0eceb3 None admin] PATH_INFO: `/v3/projects/b287589e923744ec9ed8d9c9f4196506/users/a54e7225e12b498b9af200aa54a186ad/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:50.425798 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e84ab4f-dfd7-4be2-932d-1dd03a0eceb3 None admin] RBAC: Authorizing `identity:create_grant(project_id=b287589e923744ec9ed8d9c9f4196506, user_id=a54e7225e12b498b9af200aa54a186ad, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:50.429018 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e84ab4f-dfd7-4be2-932d-1dd03a0eceb3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:50.453589 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826f9dc-b14d-4eef-b14f-b2dcb6e8ac81 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:50.491719 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a227963c-d56e-4347-9d52-f97fe3aec6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:50.492423 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a227963c-d56e-4347-9d52-f97fe3aec6ae None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:50.492701 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a227963c-d56e-4347-9d52-f97fe3aec6ae None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:50.492977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a227963c-d56e-4347-9d52-f97fe3aec6ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:50.504473 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a227963c-d56e-4347-9d52-f97fe3aec6ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:50.513926 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 129/255] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 12:22:50 2022] PUT /identity/v3/projects/b287589e923744ec9ed8d9c9f4196506/users/a54e7225e12b498b9af200aa54a186ad/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:22:50.515435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-a227963c-d56e-4347-9d52-f97fe3aec6ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:22:50.515435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:50.515435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:50.515435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:50.515435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:50.515435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:50.515435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:50.515435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:50.515435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:50.515435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:50.515435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:50.515435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:50.515435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:50.515435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:50.515435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:50.515435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:50.516347 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:50.516347 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:50.516347 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:50.516347 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:50.516347 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:50.516347 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:50.516347 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:50.516347 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:50.516347 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:50.516347 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:50.516347 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:50.516347 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:50.516347 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:50.516347 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:50.516347 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:50.516347 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:50.517525 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:50.517525 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:50.517525 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:50.517525 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:50.517525 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:50.517525 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:50.517525 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:50.517525 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:50.517525 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:50.517525 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:50.517525 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:22:50.517525 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:22:50.517525 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 127/256] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 12:22:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:50.543020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84ab4f-dfd7-4be2-932d-1dd03a0eceb3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:50.594005 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:50.633493 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e671fa-4d45-459d-afb3-40f23733557a None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:50.633714 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e671fa-4d45-459d-afb3-40f23733557a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:50.633953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e671fa-4d45-459d-afb3-40f23733557a None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:50.635640 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 128/257] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:50.656713 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d412b2-2da3-4a76-bf1b-391b0ac1194d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:50.656996 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d412b2-2da3-4a76-bf1b-391b0ac1194d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:50.661252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d412b2-2da3-4a76-bf1b-391b0ac1194d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:50.683777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84ab4f-dfd7-4be2-932d-1dd03a0eceb3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:50.697818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-93d412b2-2da3-4a76-bf1b-391b0ac1194d None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:50.699547 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-93d412b2-2da3-4a76-bf1b-391b0ac1194d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:50.757802 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c59de-f3f6-4e1c-81bd-61cc955527e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:50.758433 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c59de-f3f6-4e1c-81bd-61cc955527e6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:50.758714 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c59de-f3f6-4e1c-81bd-61cc955527e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:50.758990 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c59de-f3f6-4e1c-81bd-61cc955527e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:50.759958 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c59de-f3f6-4e1c-81bd-61cc955527e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:50.760256 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c59de-f3f6-4e1c-81bd-61cc955527e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:50.762301 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c59de-f3f6-4e1c-81bd-61cc955527e6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:50.779463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 130/258] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:22:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:50.792218 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c59de-f3f6-4e1c-81bd-61cc955527e6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:50.844091 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 129/259] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:22:50.886219 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d412b2-2da3-4a76-bf1b-391b0ac1194d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:50.975204 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c59de-f3f6-4e1c-81bd-61cc955527e6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:50.978639 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:50.982106 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:51.030810 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9803b78-4f17-4f6d-add9-c102c0777688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:51.031390 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9803b78-4f17-4f6d-add9-c102c0777688 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:51.031666 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9803b78-4f17-4f6d-add9-c102c0777688 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:51.031929 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9803b78-4f17-4f6d-add9-c102c0777688 None admin] PATH_INFO: `/v3/users/8fc8b8a221da411786fd37147e42f189` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:51.034159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9803b78-4f17-4f6d-add9-c102c0777688 None admin] RBAC: Authorizing `identity:get_user(user_id=8fc8b8a221da411786fd37147e42f189)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:51.035571 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9803b78-4f17-4f6d-add9-c102c0777688 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:51.037382 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 131/260] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:50 2022] GET /identity/v3/users/8fc8b8a221da411786fd37147e42f189 => generated 283 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:51.042789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9803b78-4f17-4f6d-add9-c102c0777688 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:51.102032 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d412b2-2da3-4a76-bf1b-391b0ac1194d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:51.160426 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9803b78-4f17-4f6d-add9-c102c0777688 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:51.179087 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b370a39-3a36-4892-9391-301db70b6564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:51.179756 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b370a39-3a36-4892-9391-301db70b6564 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:51.180034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b370a39-3a36-4892-9391-301db70b6564 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:51.180306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b370a39-3a36-4892-9391-301db70b6564 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:51.204608 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b370a39-3a36-4892-9391-301db70b6564 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:51.211035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-1b370a39-3a36-4892-9391-301db70b6564 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:22:51.211035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:51.211035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:51.211035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:51.211035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:51.211035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:51.211035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:51.211035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:51.211035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:51.211035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:51.211035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:51.211035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:51.211035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:51.211035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:51.211035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:51.211035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:51.211928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:51.211928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:51.211928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:51.211928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:51.211928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:51.211928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:51.211928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:51.211928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:51.211928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:51.211928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:51.211928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:51.211928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:51.211928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:51.211928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:51.211928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:51.211928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:51.212740 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:51.212740 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:51.212740 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:51.212740 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:51.212740 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:51.212740 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:51.212740 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:51.212740 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:51.212740 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:51.212740 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:51.212740 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:22:51.212740 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:22:51.212740 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 130/261] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:22:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:22:51.220141 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b370a39-3a36-4892-9391-301db70b6564 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:51.221097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f567621-f66e-4a66-9b3a-490d1177e2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:51.221702 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f567621-f66e-4a66-9b3a-490d1177e2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:51.221978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f567621-f66e-4a66-9b3a-490d1177e2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:51.222272 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f567621-f66e-4a66-9b3a-490d1177e2e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:51.283577 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:51.356256 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f567621-f66e-4a66-9b3a-490d1177e2e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:51.358085 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f567621-f66e-4a66-9b3a-490d1177e2e8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:51.421735 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b370a39-3a36-4892-9391-301db70b6564 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:51.435541 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:51.479528 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e71d42-8c88-4f30-9f2f-1a624af05b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:51.480332 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e71d42-8c88-4f30-9f2f-1a624af05b28 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:51.480582 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e71d42-8c88-4f30-9f2f-1a624af05b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:51.480858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e71d42-8c88-4f30-9f2f-1a624af05b28 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:51.481538 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e71d42-8c88-4f30-9f2f-1a624af05b28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:51.481843 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e71d42-8c88-4f30-9f2f-1a624af05b28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:51.483308 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e71d42-8c88-4f30-9f2f-1a624af05b28 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:51.494350 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 131/262] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:51.504371 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e71d42-8c88-4f30-9f2f-1a624af05b28 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:51.603112 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 132/263] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:22:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:22:51.608644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f567621-f66e-4a66-9b3a-490d1177e2e8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:51.703530 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e71d42-8c88-4f30-9f2f-1a624af05b28 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:51.728198 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:51.751913 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44f42b-60b3-4ea7-adf5-d1ed156bcbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:51.752547 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44f42b-60b3-4ea7-adf5-d1ed156bcbc3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:51.752822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44f42b-60b3-4ea7-adf5-d1ed156bcbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:51.753092 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44f42b-60b3-4ea7-adf5-d1ed156bcbc3 None admin] PATH_INFO: `/v3/users/8fc8b8a221da411786fd37147e42f189` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:51.754484 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44f42b-60b3-4ea7-adf5-d1ed156bcbc3 None admin] RBAC: Authorizing `identity:get_user(user_id=8fc8b8a221da411786fd37147e42f189)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:51.755939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44f42b-60b3-4ea7-adf5-d1ed156bcbc3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:51.763614 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 132/264] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:51 2022] GET /identity/v3/users/8fc8b8a221da411786fd37147e42f189 => generated 283 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:51.768058 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44f42b-60b3-4ea7-adf5-d1ed156bcbc3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:51.794375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f567621-f66e-4a66-9b3a-490d1177e2e8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:51.808872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:51.859070 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70ad30-2661-494f-a2a9-84068e2b4a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:51.859811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70ad30-2661-494f-a2a9-84068e2b4a61 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:51.860090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70ad30-2661-494f-a2a9-84068e2b4a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:51.860360 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70ad30-2661-494f-a2a9-84068e2b4a61 None admin] PATH_INFO: `/v3/projects/b287589e923744ec9ed8d9c9f4196506` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:51.861479 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70ad30-2661-494f-a2a9-84068e2b4a61 None admin] RBAC: Authorizing `identity:get_project(project_id=b287589e923744ec9ed8d9c9f4196506)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:51.862930 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70ad30-2661-494f-a2a9-84068e2b4a61 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:51.868222 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 133/265] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:22:51 2022] GET /identity/v3/projects/b287589e923744ec9ed8d9c9f4196506 => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:51.871296 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70ad30-2661-494f-a2a9-84068e2b4a61 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:51.893621 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44f42b-60b3-4ea7-adf5-d1ed156bcbc3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:51.933950 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3cbfef-345e-4632-bcec-75f60aecfb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:51.934562 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3cbfef-345e-4632-bcec-75f60aecfb85 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:51.934838 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3cbfef-345e-4632-bcec-75f60aecfb85 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:51.935109 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3cbfef-345e-4632-bcec-75f60aecfb85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:52.020575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70ad30-2661-494f-a2a9-84068e2b4a61 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:52.059617 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7951b4a-3e39-4a0d-93c8-7dbbb0632fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:52.060715 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7951b4a-3e39-4a0d-93c8-7dbbb0632fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:52.061201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7951b4a-3e39-4a0d-93c8-7dbbb0632fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:52.061669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7951b4a-3e39-4a0d-93c8-7dbbb0632fe5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:52.062611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7951b4a-3e39-4a0d-93c8-7dbbb0632fe5 None admin] RBAC: Adding query filter params (role.id=0f58d2782a2840648bbbc2ab26ec02df, scope.project.id=b287589e923744ec9ed8d9c9f4196506, user.id=8fc8b8a221da411786fd37147e42f189) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:52.063117 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7951b4a-3e39-4a0d-93c8-7dbbb0632fe5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:52.065286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7951b4a-3e39-4a0d-93c8-7dbbb0632fe5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:52.067307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:52.073042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 134/266] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:22:51 2022] GET /identity/v3/role_assignments?user.id=8fc8b8a221da411786fd37147e42f189&scope.project.id=b287589e923744ec9ed8d9c9f4196506&role.id=0f58d2782a2840648bbbc2ab26ec02df => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:52.096334 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3cbfef-345e-4632-bcec-75f60aecfb85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:52.097879 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3cbfef-345e-4632-bcec-75f60aecfb85 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:52.161589 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:52.254269 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 133/267] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:22:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:22:52.265305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7951b4a-3e39-4a0d-93c8-7dbbb0632fe5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:52.390327 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7951b4a-3e39-4a0d-93c8-7dbbb0632fe5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:52.430951 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad35aa8-0bd9-4520-9b45-66717653ec59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:52.431616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad35aa8-0bd9-4520-9b45-66717653ec59 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:52.431893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad35aa8-0bd9-4520-9b45-66717653ec59 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:52.432169 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad35aa8-0bd9-4520-9b45-66717653ec59 None admin] PATH_INFO: `/v3/projects/b287589e923744ec9ed8d9c9f4196506` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:52.433317 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad35aa8-0bd9-4520-9b45-66717653ec59 None admin] RBAC: Authorizing `identity:get_project(project_id=b287589e923744ec9ed8d9c9f4196506)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:52.434769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad35aa8-0bd9-4520-9b45-66717653ec59 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:52.436753 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 135/268] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:22:52 2022] GET /identity/v3/projects/b287589e923744ec9ed8d9c9f4196506 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:52.442435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3cbfef-345e-4632-bcec-75f60aecfb85 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:52.575435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3cbfef-345e-4632-bcec-75f60aecfb85 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:52.624009 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0674a15-d995-4957-a46c-3a4ff8f34533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:52.624617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0674a15-d995-4957-a46c-3a4ff8f34533 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:52.624891 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0674a15-d995-4957-a46c-3a4ff8f34533 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:52.625163 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0674a15-d995-4957-a46c-3a4ff8f34533 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:52.625947 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0674a15-d995-4957-a46c-3a4ff8f34533 None admin] RBAC: Adding query filter params (role.id=bffbe227cda1418cbb58394ddff6f861, scope.project.id=b287589e923744ec9ed8d9c9f4196506, user.id=8fc8b8a221da411786fd37147e42f189) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:52.626254 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0674a15-d995-4957-a46c-3a4ff8f34533 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:52.627737 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0674a15-d995-4957-a46c-3a4ff8f34533 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:52.629524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:52.639896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 134/269] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:22:52 2022] GET /identity/v3/role_assignments?user.id=8fc8b8a221da411786fd37147e42f189&scope.project.id=b287589e923744ec9ed8d9c9f4196506&role.id=bffbe227cda1418cbb58394ddff6f861 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:52.746995 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c3040-7dac-430d-8700-e8434a000758 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:52.747563 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c3040-7dac-430d-8700-e8434a000758 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:52.747640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c3040-7dac-430d-8700-e8434a000758 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:52.749249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 136/270] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:52.754486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8dc27e-3685-495d-a00c-5a580ca0ae7e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:52.754774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8dc27e-3685-495d-a00c-5a580ca0ae7e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:52.755048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8dc27e-3685-495d-a00c-5a580ca0ae7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:52.778505 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7b8dc27e-3685-495d-a00c-5a580ca0ae7e None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:52.779732 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7b8dc27e-3685-495d-a00c-5a580ca0ae7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:52.839993 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f31f0-226f-4b3c-8704-850ace4fcdf8 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:52.840386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f31f0-226f-4b3c-8704-850ace4fcdf8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:52.840758 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f31f0-226f-4b3c-8704-850ace4fcdf8 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:52.843283 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 137/271] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:22:52.848233 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd17d12-6876-44d5-9bd2-9c2677ff10d5 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:52.848609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd17d12-6876-44d5-9bd2-9c2677ff10d5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:52.848956 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd17d12-6876-44d5-9bd2-9c2677ff10d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:52.868920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-bdd17d12-6876-44d5-9bd2-9c2677ff10d5 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:52.870499 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bdd17d12-6876-44d5-9bd2-9c2677ff10d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:52.884950 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 135/272] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:52.933289 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8dc27e-3685-495d-a00c-5a580ca0ae7e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:52.969836 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:52.983053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 138/273] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:22:53.006754 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd17d12-6876-44d5-9bd2-9c2677ff10d5 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:53.072347 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8dc27e-3685-495d-a00c-5a580ca0ae7e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:53.105366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:53.114281 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d948d-c063-430d-81f3-05ae91f4ceda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:53.115019 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d948d-c063-430d-81f3-05ae91f4ceda None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:53.115904 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d948d-c063-430d-81f3-05ae91f4ceda None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:53.121628 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d948d-c063-430d-81f3-05ae91f4ceda None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:53.128637 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d948d-c063-430d-81f3-05ae91f4ceda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:53.131122 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd17d12-6876-44d5-9bd2-9c2677ff10d5 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:53.135535 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-873d948d-c063-430d-81f3-05ae91f4ceda None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:22:53.135535 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:53.135535 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:53.135535 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:53.135535 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:53.135535 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:53.135535 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:53.135535 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:53.135535 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:53.135535 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:53.135535 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:53.135535 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:53.135535 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:53.135535 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:53.135535 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:53.135535 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:53.136561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:53.136561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:53.136561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:53.136561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:53.136561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:53.136561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:53.136561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:53.136561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:53.136561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:53.136561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:53.136561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:53.136561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:53.136561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:53.136561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:53.136561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:53.136561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:53.137345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:53.137345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:53.137345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:53.137345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:53.137345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:53.137345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:53.137345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:53.137345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:53.137345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:53.137345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:53.137345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:22:53.137345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:22:53.142047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 136/274] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:53.143515 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d948d-c063-430d-81f3-05ae91f4ceda None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:53.180585 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f41552-a764-4e7f-a81f-eff07d103c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:53.181514 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f41552-a764-4e7f-a81f-eff07d103c6d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:53.182029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f41552-a764-4e7f-a81f-eff07d103c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:53.182532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f41552-a764-4e7f-a81f-eff07d103c6d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:53.183803 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f41552-a764-4e7f-a81f-eff07d103c6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:53.185229 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f41552-a764-4e7f-a81f-eff07d103c6d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:53.186774 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 139/275] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:22:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:53.192676 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f41552-a764-4e7f-a81f-eff07d103c6d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:53.268867 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:53.282531 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d948d-c063-430d-81f3-05ae91f4ceda None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:53.320937 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f41552-a764-4e7f-a81f-eff07d103c6d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:53.334796 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2027259-e9b6-420a-9b2b-6e31309abe47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:53.335971 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2027259-e9b6-420a-9b2b-6e31309abe47 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:53.336427 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2027259-e9b6-420a-9b2b-6e31309abe47 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:53.336903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2027259-e9b6-420a-9b2b-6e31309abe47 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:53.337766 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2027259-e9b6-420a-9b2b-6e31309abe47 None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:53.338373 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2027259-e9b6-420a-9b2b-6e31309abe47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:53.340184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2027259-e9b6-420a-9b2b-6e31309abe47 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:53.358646 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 137/276] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 12:22:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:53.360403 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2027259-e9b6-420a-9b2b-6e31309abe47 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:53.362487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c6f87-d234-4532-b782-702eb2867532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:53.363475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c6f87-d234-4532-b782-702eb2867532 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:53.364101 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c6f87-d234-4532-b782-702eb2867532 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:53.364549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c6f87-d234-4532-b782-702eb2867532 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:53.365402 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c6f87-d234-4532-b782-702eb2867532 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:53.367076 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c6f87-d234-4532-b782-702eb2867532 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:53.415855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:53.467413 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2027259-e9b6-420a-9b2b-6e31309abe47 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:53.473414 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 140/277] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 12:22:53 2022] POST /identity/v3/users => generated 293 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:22:53.473414 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e287961-4cee-4eeb-a148-d6b379ef396d None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:53.473414 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e287961-4cee-4eeb-a148-d6b379ef396d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:53.473414 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e287961-4cee-4eeb-a148-d6b379ef396d None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:53.473414 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 141/278] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:53.478047 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fe407-4644-4b12-86de-2067f2639c10 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:53.478323 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fe407-4644-4b12-86de-2067f2639c10 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:53.479565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fe407-4644-4b12-86de-2067f2639c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:53.501700 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-004fe407-4644-4b12-86de-2067f2639c10 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:53.504091 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-004fe407-4644-4b12-86de-2067f2639c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:53.516329 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b2330-1ef3-4536-86ec-d4616a035d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:53.516982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b2330-1ef3-4536-86ec-d4616a035d7b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:53.517315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b2330-1ef3-4536-86ec-d4616a035d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:53.517594 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b2330-1ef3-4536-86ec-d4616a035d7b None admin] PATH_INFO: `/v3/users/6af8df8542514fd0aeb9b903b993bae5` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:53.533245 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b2330-1ef3-4536-86ec-d4616a035d7b None admin] RBAC: Authorizing `identity:get_user(user_id=6af8df8542514fd0aeb9b903b993bae5)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:53.534715 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b2330-1ef3-4536-86ec-d4616a035d7b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:53.536820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 138/279] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:53 2022] GET /identity/v3/users/6af8df8542514fd0aeb9b903b993bae5 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:53.543693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b2330-1ef3-4536-86ec-d4616a035d7b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:53.591465 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 142/280] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:22:53.596777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b87730-3009-4663-9f02-4afddf7f976b None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:53.597065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b87730-3009-4663-9f02-4afddf7f976b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:53.597343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b87730-3009-4663-9f02-4afddf7f976b None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:53.598873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 143/281] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:53.605623 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783c693-31e4-4162-9733-bbd9c570a50c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:53.605962 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783c693-31e4-4162-9733-bbd9c570a50c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:53.606162 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783c693-31e4-4162-9733-bbd9c570a50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:53.627512 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6783c693-31e4-4162-9733-bbd9c570a50c None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:53.628728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6783c693-31e4-4162-9733-bbd9c570a50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:53.670638 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b2330-1ef3-4536-86ec-d4616a035d7b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:53.682958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:53.684972 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:53.710054 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 144/282] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:22:53.712087 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6783c693-31e4-4162-9733-bbd9c570a50c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:53.715464 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3f819-17b6-4ecf-b651-d6df600516db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:53.716088 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3f819-17b6-4ecf-b651-d6df600516db None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:53.716367 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3f819-17b6-4ecf-b651-d6df600516db None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:53.716638 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3f819-17b6-4ecf-b651-d6df600516db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:53.717360 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3f819-17b6-4ecf-b651-d6df600516db None admin] RBAC: Adding query filter params (role.id=872ba0c4212a4684b539a68fc2c20b25, user.id=6af8df8542514fd0aeb9b903b993bae5, scope.system=all) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:53.717670 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3f819-17b6-4ecf-b651-d6df600516db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:53.719103 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3f819-17b6-4ecf-b651-d6df600516db None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:53.725162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 139/283] 10.222.0.49 () {56 vars in 1308 bytes} [Wed Aug 10 12:22:53 2022] GET /identity/v3/role_assignments?user.id=6af8df8542514fd0aeb9b903b993bae5&scope.system=all&role.id=872ba0c4212a4684b539a68fc2c20b25 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:53.734317 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3f819-17b6-4ecf-b651-d6df600516db None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:53.861874 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6783c693-31e4-4162-9733-bbd9c570a50c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:53.880044 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3f819-17b6-4ecf-b651-d6df600516db None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:53.913197 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05dcf0-ee45-4458-aeba-a86c804eff67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:53.913871 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05dcf0-ee45-4458-aeba-a86c804eff67 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:53.914357 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05dcf0-ee45-4458-aeba-a86c804eff67 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:53.914709 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05dcf0-ee45-4458-aeba-a86c804eff67 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:53.925164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b80906-7b6a-43fd-a5e0-bc1a4ef91fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:53.925803 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b80906-7b6a-43fd-a5e0-bc1a4ef91fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:53.926077 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b80906-7b6a-43fd-a5e0-bc1a4ef91fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:53.926348 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b80906-7b6a-43fd-a5e0-bc1a4ef91fa2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:53.927372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05dcf0-ee45-4458-aeba-a86c804eff67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:53.933270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-9f05dcf0-ee45-4458-aeba-a86c804eff67 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:22:53.933270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:53.933270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:53.933270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:53.933270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:53.933270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:53.933270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:53.933270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:53.933270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:53.933270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:53.933270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:53.933270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:53.933270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:53.933270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:53.933270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:53.933270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:53.934169 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:53.934169 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:53.934169 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:53.934169 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:53.934169 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:53.934169 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:53.934169 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:53.934169 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:53.934169 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:53.934169 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:53.934169 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:53.934169 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:53.934169 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:53.934169 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:53.934169 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:53.934169 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:53.935095 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:53.935095 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:53.935095 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:53.935095 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:53.935095 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:53.935095 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:53.935095 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:53.935095 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:53.935095 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:53.935095 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:53.935095 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:22:53.935095 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:22:53.935095 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 145/284] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:53.940628 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b80906-7b6a-43fd-a5e0-bc1a4ef91fa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:53.941724 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05dcf0-ee45-4458-aeba-a86c804eff67 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:53.948828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-09b80906-7b6a-43fd-a5e0-bc1a4ef91fa2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:22:53.948828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:53.948828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:53.948828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:53.948828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:53.948828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:53.948828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:53.948828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:53.948828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:53.948828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:53.948828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:53.948828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:53.948828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:53.948828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:53.948828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:53.948828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:53.951441 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:53.951441 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:53.951441 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:53.951441 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:53.951441 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:53.951441 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:53.951441 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:53.951441 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:53.951441 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:53.951441 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:53.951441 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:53.951441 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:53.951441 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:53.951441 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:53.951441 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:53.951441 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:53.952888 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:53.952888 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:53.952888 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:53.952888 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:53.952888 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:53.952888 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:53.952888 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:53.952888 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:53.952888 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:53.952888 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:53.952888 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:22:53.952888 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:22:53.952888 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 140/285] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:53.961632 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b80906-7b6a-43fd-a5e0-bc1a4ef91fa2 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:53.997554 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:53.999505 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:54.062366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05dcf0-ee45-4458-aeba-a86c804eff67 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:54.067368 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b80906-7b6a-43fd-a5e0-bc1a4ef91fa2 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:54.101284 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2dfa41-23ae-4e3c-931b-337b601e3121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:54.102271 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2dfa41-23ae-4e3c-931b-337b601e3121 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:54.102744 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2dfa41-23ae-4e3c-931b-337b601e3121 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:54.103243 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2dfa41-23ae-4e3c-931b-337b601e3121 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:54.104250 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2dfa41-23ae-4e3c-931b-337b601e3121 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:54.104745 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2dfa41-23ae-4e3c-931b-337b601e3121 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:54.106355 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2dfa41-23ae-4e3c-931b-337b601e3121 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:54.116821 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 146/286] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 12:22:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:54.122817 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2dfa41-23ae-4e3c-931b-337b601e3121 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:54.126596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd2308-05f4-49b1-95b2-3ba88d91c116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:54.129423 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd2308-05f4-49b1-95b2-3ba88d91c116 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:54.129939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd2308-05f4-49b1-95b2-3ba88d91c116 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:54.132930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd2308-05f4-49b1-95b2-3ba88d91c116 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:54.133840 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd2308-05f4-49b1-95b2-3ba88d91c116 None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:54.134341 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd2308-05f4-49b1-95b2-3ba88d91c116 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:54.136666 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd2308-05f4-49b1-95b2-3ba88d91c116 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:54.154891 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 141/287] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 12:22:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:54.161381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd2308-05f4-49b1-95b2-3ba88d91c116 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:54.346849 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2dfa41-23ae-4e3c-931b-337b601e3121 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:54.349368 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:54.352578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:54.365282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd2308-05f4-49b1-95b2-3ba88d91c116 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:54.410392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddc9c7-49f5-40fe-9064-374f75bdba65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:54.410972 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddc9c7-49f5-40fe-9064-374f75bdba65 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:54.411315 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddc9c7-49f5-40fe-9064-374f75bdba65 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:54.411587 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddc9c7-49f5-40fe-9064-374f75bdba65 None admin] PATH_INFO: `/v3/users/67d9ac49973a425686988215f938c364` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:54.413965 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ddc9c7-49f5-40fe-9064-374f75bdba65 None admin] RBAC: Authorizing `identity:get_user(user_id=67d9ac49973a425686988215f938c364)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:54.415421 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ddc9c7-49f5-40fe-9064-374f75bdba65 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:54.418729 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 147/288] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:54 2022] GET /identity/v3/users/67d9ac49973a425686988215f938c364 => generated 297 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:54.425297 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddc9c7-49f5-40fe-9064-374f75bdba65 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:54.433839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bdb482-3f73-4edd-9557-dfaaf48c9ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:54.434481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bdb482-3f73-4edd-9557-dfaaf48c9ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:54.434754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bdb482-3f73-4edd-9557-dfaaf48c9ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:54.435023 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bdb482-3f73-4edd-9557-dfaaf48c9ad9 None admin] PATH_INFO: `/v3/users/a54e7225e12b498b9af200aa54a186ad` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:54.440611 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bdb482-3f73-4edd-9557-dfaaf48c9ad9 None admin] RBAC: Authorizing `identity:get_user(user_id=a54e7225e12b498b9af200aa54a186ad)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:54.442046 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bdb482-3f73-4edd-9557-dfaaf48c9ad9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:54.444543 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 142/289] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:54 2022] GET /identity/v3/users/a54e7225e12b498b9af200aa54a186ad => generated 297 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:54.451301 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bdb482-3f73-4edd-9557-dfaaf48c9ad9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:54.622102 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddc9c7-49f5-40fe-9064-374f75bdba65 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:54.661635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efccc6c-3d2e-4046-8869-8b2a4f7b4b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:54.662368 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efccc6c-3d2e-4046-8869-8b2a4f7b4b82 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:54.662951 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efccc6c-3d2e-4046-8869-8b2a4f7b4b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:54.663030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efccc6c-3d2e-4046-8869-8b2a4f7b4b82 None admin] PATH_INFO: `/v3/projects/b287589e923744ec9ed8d9c9f4196506` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:54.664209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efccc6c-3d2e-4046-8869-8b2a4f7b4b82 None admin] RBAC: Authorizing `identity:get_project(project_id=b287589e923744ec9ed8d9c9f4196506)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:54.665648 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efccc6c-3d2e-4046-8869-8b2a4f7b4b82 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:54.667593 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 148/290] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:22:54 2022] GET /identity/v3/projects/b287589e923744ec9ed8d9c9f4196506 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:54.669151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bdb482-3f73-4edd-9557-dfaaf48c9ad9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:54.673099 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efccc6c-3d2e-4046-8869-8b2a4f7b4b82 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:54.727565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc18984-b70d-47e9-9150-942eec1f28ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:54.728228 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc18984-b70d-47e9-9150-942eec1f28ef None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:54.728437 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc18984-b70d-47e9-9150-942eec1f28ef None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:54.728708 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc18984-b70d-47e9-9150-942eec1f28ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:54.737074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:54.784723 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:54.831851 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc18984-b70d-47e9-9150-942eec1f28ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:54.833312 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc18984-b70d-47e9-9150-942eec1f28ef None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:54.854485 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efccc6c-3d2e-4046-8869-8b2a4f7b4b82 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:54.924883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006d207-5ee7-4dea-bbbb-a9ce11e2c5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:54.925601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006d207-5ee7-4dea-bbbb-a9ce11e2c5b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:54.925792 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006d207-5ee7-4dea-bbbb-a9ce11e2c5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:54.926065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006d207-5ee7-4dea-bbbb-a9ce11e2c5b7 None admin] PATH_INFO: `/v3/projects/b287589e923744ec9ed8d9c9f4196506/users/67d9ac49973a425686988215f938c364/roles/78c5ea5c9f154c6da4912cfc338b6474` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:54.929245 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006d207-5ee7-4dea-bbbb-a9ce11e2c5b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=b287589e923744ec9ed8d9c9f4196506, user_id=67d9ac49973a425686988215f938c364, role_id=78c5ea5c9f154c6da4912cfc338b6474)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:54.930685 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006d207-5ee7-4dea-bbbb-a9ce11e2c5b7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:54.989082 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 143/291] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:22:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:22:54.995646 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc18984-b70d-47e9-9150-942eec1f28ef None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:55.028291 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 149/292] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 12:22:54 2022] PUT /identity/v3/projects/b287589e923744ec9ed8d9c9f4196506/users/67d9ac49973a425686988215f938c364/roles/78c5ea5c9f154c6da4912cfc338b6474 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:22:55.263209 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:55.312871 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc18984-b70d-47e9-9150-942eec1f28ef None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:55.351600 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030e77e-a0ad-48d8-8e7c-f1fffae2f1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:55.352232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030e77e-a0ad-48d8-8e7c-f1fffae2f1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:55.352507 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030e77e-a0ad-48d8-8e7c-f1fffae2f1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:55.352781 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030e77e-a0ad-48d8-8e7c-f1fffae2f1a1 None admin] PATH_INFO: `/v3/projects/b287589e923744ec9ed8d9c9f4196506` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:55.353897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030e77e-a0ad-48d8-8e7c-f1fffae2f1a1 None admin] RBAC: Authorizing `identity:get_project(project_id=b287589e923744ec9ed8d9c9f4196506)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:55.355353 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030e77e-a0ad-48d8-8e7c-f1fffae2f1a1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:55.357270 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 144/293] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:22:54 2022] GET /identity/v3/projects/b287589e923744ec9ed8d9c9f4196506 => generated 302 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:55.363170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006d207-5ee7-4dea-bbbb-a9ce11e2c5b7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:55.510420 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:55.514468 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006d207-5ee7-4dea-bbbb-a9ce11e2c5b7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:55.564001 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68599fb2-969f-40d1-9734-2a8d8ef08716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:55.564620 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68599fb2-969f-40d1-9734-2a8d8ef08716 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:55.564899 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68599fb2-969f-40d1-9734-2a8d8ef08716 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:55.565170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68599fb2-969f-40d1-9734-2a8d8ef08716 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:55.565898 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68599fb2-969f-40d1-9734-2a8d8ef08716 None admin] RBAC: Adding query filter params (role.id=872ba0c4212a4684b539a68fc2c20b25, scope.project.id=b287589e923744ec9ed8d9c9f4196506, user.id=a54e7225e12b498b9af200aa54a186ad) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:55.566209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68599fb2-969f-40d1-9734-2a8d8ef08716 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:55.567750 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68599fb2-969f-40d1-9734-2a8d8ef08716 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:55.576736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 150/294] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:22:55 2022] GET /identity/v3/role_assignments?user.id=a54e7225e12b498b9af200aa54a186ad&scope.project.id=b287589e923744ec9ed8d9c9f4196506&role.id=872ba0c4212a4684b539a68fc2c20b25 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:56.002565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f7268-11ac-4910-b137-b0f86571314b None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:56.003567 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f7268-11ac-4910-b137-b0f86571314b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:56.004054 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f7268-11ac-4910-b137-b0f86571314b None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:56.005915 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 145/295] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:56.011983 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f9bcc-6241-4e50-83dc-7aa94ccb5920 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:56.012846 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f9bcc-6241-4e50-83dc-7aa94ccb5920 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:56.013317 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f9bcc-6241-4e50-83dc-7aa94ccb5920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:56.028804 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd698c-efee-435e-8540-d75582a815d1 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:56.029131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd698c-efee-435e-8540-d75582a815d1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:56.029365 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd698c-efee-435e-8540-d75582a815d1 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:56.031673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 146/296] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:22:56.037041 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-068f9bcc-6241-4e50-83dc-7aa94ccb5920 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:56.037162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacf135-7c36-44a0-9707-b190295556df None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:56.037447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacf135-7c36-44a0-9707-b190295556df None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:56.037872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacf135-7c36-44a0-9707-b190295556df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:56.039158 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-068f9bcc-6241-4e50-83dc-7aa94ccb5920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:56.059436 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-aaacf135-7c36-44a0-9707-b190295556df None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:56.060614 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-aaacf135-7c36-44a0-9707-b190295556df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:56.139280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 147/297] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:56.144722 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 151/298] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:56.158247 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaacf135-7c36-44a0-9707-b190295556df None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:56.166385 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f9bcc-6241-4e50-83dc-7aa94ccb5920 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:56.220885 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:56.263798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaacf135-7c36-44a0-9707-b190295556df None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:56.292467 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:56.304565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f9bcc-6241-4e50-83dc-7aa94ccb5920 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:56.309540 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2ddfa-0d32-439d-b431-56d311454950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:56.310316 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2ddfa-0d32-439d-b431-56d311454950 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:56.310656 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2ddfa-0d32-439d-b431-56d311454950 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:56.311037 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2ddfa-0d32-439d-b431-56d311454950 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:56.324628 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd2ddfa-0d32-439d-b431-56d311454950 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:56.329872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-fdd2ddfa-0d32-439d-b431-56d311454950 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:22:56.329872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:56.329872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:56.329872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:56.329872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:56.329872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:56.329872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:56.329872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:56.329872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:56.329872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:56.329872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:56.329872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:56.329872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:56.329872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:56.329872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:56.329872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:56.330820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:56.330820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:56.330820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:56.330820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:56.330820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:56.330820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:56.330820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:56.330820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:56.330820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:56.330820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:56.330820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:56.330820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:56.330820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:56.330820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:56.330820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:56.330820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:56.331672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:56.331672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:56.331672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:56.331672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:56.331672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:56.331672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:56.331672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:56.331672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:56.331672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:56.331672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:56.331672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:22:56.331672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:22:56.335359 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 148/299] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:56.337068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed50623-28e9-42f3-a4ce-641f7046656f None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:56.337360 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed50623-28e9-42f3-a4ce-641f7046656f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:56.337743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed50623-28e9-42f3-a4ce-641f7046656f None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:56.339230 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 149/300] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:56.343844 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed50623-28e9-42f3-a4ce-641f7046656f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:56.346470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c14973-ce33-45ac-b727-81b8a123a13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:56.347314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c14973-ce33-45ac-b727-81b8a123a13d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:56.347597 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c14973-ce33-45ac-b727-81b8a123a13d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:56.348682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c14973-ce33-45ac-b727-81b8a123a13d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:56.350431 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c14973-ce33-45ac-b727-81b8a123a13d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:56.352058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c14973-ce33-45ac-b727-81b8a123a13d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:56.354170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 152/301] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:22:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:56.359484 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c43dd7-0e5a-4ef9-a59b-cf9881b1f657 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:56.359484 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c43dd7-0e5a-4ef9-a59b-cf9881b1f657 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:56.359484 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c43dd7-0e5a-4ef9-a59b-cf9881b1f657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:56.378264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-60c43dd7-0e5a-4ef9-a59b-cf9881b1f657 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:56.379489 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-60c43dd7-0e5a-4ef9-a59b-cf9881b1f657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:56.451376 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 153/302] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:56.452588 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c43dd7-0e5a-4ef9-a59b-cf9881b1f657 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:56.463490 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed50623-28e9-42f3-a4ce-641f7046656f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:56.486552 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:56.500924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63143201-8733-42dd-ab08-601a467cba9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:56.501532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63143201-8733-42dd-ab08-601a467cba9b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:56.501811 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63143201-8733-42dd-ab08-601a467cba9b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:56.502259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63143201-8733-42dd-ab08-601a467cba9b None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:56.502749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63143201-8733-42dd-ab08-601a467cba9b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:56.503056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63143201-8733-42dd-ab08-601a467cba9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:56.504464 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63143201-8733-42dd-ab08-601a467cba9b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:56.517142 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 150/303] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 12:22:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:56.517142 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63143201-8733-42dd-ab08-601a467cba9b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:56.552769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:56.563397 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c43dd7-0e5a-4ef9-a59b-cf9881b1f657 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:56.600476 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5a7b8-0c9d-4602-b5b6-085d3740151b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:56.601899 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5a7b8-0c9d-4602-b5b6-085d3740151b None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:56.602192 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5a7b8-0c9d-4602-b5b6-085d3740151b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:56.602469 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5a7b8-0c9d-4602-b5b6-085d3740151b None admin] PATH_INFO: `/v3/groups` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:56.603079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a5a7b8-0c9d-4602-b5b6-085d3740151b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:56.604568 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a5a7b8-0c9d-4602-b5b6-085d3740151b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:56.623790 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63143201-8733-42dd-ab08-601a467cba9b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:56.657879 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93129dc8-a85d-4fda-ab7b-01b786dba718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:56.658511 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93129dc8-a85d-4fda-ab7b-01b786dba718 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:56.658787 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93129dc8-a85d-4fda-ab7b-01b786dba718 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:56.659058 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93129dc8-a85d-4fda-ab7b-01b786dba718 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:56.670203 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93129dc8-a85d-4fda-ab7b-01b786dba718 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:56.675504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-93129dc8-a85d-4fda-ab7b-01b786dba718 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:22:56.675504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:56.675504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:56.675504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:56.675504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:56.675504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:56.675504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:56.675504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:56.675504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:56.675504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:56.675504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:56.675504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:56.675504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:56.675504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:56.675504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:56.675504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:56.676625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:56.676625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:56.676625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:56.676625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:56.676625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:56.676625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:56.676625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:56.676625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:56.676625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:56.676625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:56.676625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:56.676625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:56.676625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:56.676625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:56.676625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:56.676625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:56.682237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:56.682237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:56.682237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:56.682237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:56.682237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:56.682237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:56.682237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:56.682237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:56.682237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:56.682237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:56.682237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:22:56.682237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:22:56.682237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 151/304] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:56.682237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93129dc8-a85d-4fda-ab7b-01b786dba718 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:56.688589 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 154/305] 10.222.0.49 () {60 vars in 1134 bytes} [Wed Aug 10 12:22:56 2022] POST /identity/v3/groups => generated 229 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:22:56.689524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5a7b8-0c9d-4602-b5b6-085d3740151b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:56.750445 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:56.841711 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5a7b8-0c9d-4602-b5b6-085d3740151b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:56.872359 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93129dc8-a85d-4fda-ab7b-01b786dba718 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:56.901156 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc2c2f-8fa8-4688-a2f3-4900cac9d7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:56.902104 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc2c2f-8fa8-4688-a2f3-4900cac9d7b5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:56.902558 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc2c2f-8fa8-4688-a2f3-4900cac9d7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:56.903006 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc2c2f-8fa8-4688-a2f3-4900cac9d7b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:56.904161 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacc2c2f-8fa8-4688-a2f3-4900cac9d7b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:56.904658 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacc2c2f-8fa8-4688-a2f3-4900cac9d7b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:56.906994 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacc2c2f-8fa8-4688-a2f3-4900cac9d7b5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:56.922985 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 155/306] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 12:22:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:56.926371 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c61a3-86e5-4cb3-a629-a947c3af9e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:56.927014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c61a3-86e5-4cb3-a629-a947c3af9e5c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:56.927359 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c61a3-86e5-4cb3-a629-a947c3af9e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:56.927634 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c61a3-86e5-4cb3-a629-a947c3af9e5c None admin] PATH_INFO: `/v3/users/27dbe202509f4efe998a9b7b49028df3` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:56.930071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc2c2f-8fa8-4688-a2f3-4900cac9d7b5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:56.946031 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6c61a3-86e5-4cb3-a629-a947c3af9e5c None admin] RBAC: Authorizing `identity:get_user(user_id=27dbe202509f4efe998a9b7b49028df3)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:56.947667 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6c61a3-86e5-4cb3-a629-a947c3af9e5c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:56.949563 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 152/307] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:56 2022] GET /identity/v3/users/27dbe202509f4efe998a9b7b49028df3 => generated 293 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:56.954071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:56.956396 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c61a3-86e5-4cb3-a629-a947c3af9e5c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:57.071541 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc2c2f-8fa8-4688-a2f3-4900cac9d7b5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:57.078046 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:57.081527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c61a3-86e5-4cb3-a629-a947c3af9e5c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:57.110694 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e3e59-a2ea-4417-9327-37cfb1b138e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:57.111314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e3e59-a2ea-4417-9327-37cfb1b138e1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:57.111594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e3e59-a2ea-4417-9327-37cfb1b138e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:57.111865 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e3e59-a2ea-4417-9327-37cfb1b138e1 None admin] PATH_INFO: `/v3/users/6af8df8542514fd0aeb9b903b993bae5` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:57.113112 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e3e59-a2ea-4417-9327-37cfb1b138e1 None admin] RBAC: Authorizing `identity:get_user(user_id=6af8df8542514fd0aeb9b903b993bae5)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:57.114675 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e3e59-a2ea-4417-9327-37cfb1b138e1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:57.117689 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 156/308] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:56 2022] GET /identity/v3/users/6af8df8542514fd0aeb9b903b993bae5 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:57.122886 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e3e59-a2ea-4417-9327-37cfb1b138e1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:57.135952 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0fe1dc-82ec-4cad-b959-22e20508b870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:57.137204 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fe1dc-82ec-4cad-b959-22e20508b870 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:57.137684 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fe1dc-82ec-4cad-b959-22e20508b870 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:57.138155 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fe1dc-82ec-4cad-b959-22e20508b870 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:57.139611 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0fe1dc-82ec-4cad-b959-22e20508b870 None admin] RBAC: Adding query filter params (role.id=78c5ea5c9f154c6da4912cfc338b6474, user.id=27dbe202509f4efe998a9b7b49028df3, scope.system=all) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:57.140124 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0fe1dc-82ec-4cad-b959-22e20508b870 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:57.142155 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0fe1dc-82ec-4cad-b959-22e20508b870 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:57.155496 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 153/309] 10.222.0.49 () {56 vars in 1308 bytes} [Wed Aug 10 12:22:56 2022] GET /identity/v3/role_assignments?user.id=27dbe202509f4efe998a9b7b49028df3&scope.system=all&role.id=78c5ea5c9f154c6da4912cfc338b6474 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:57.221288 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:57.242663 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e3e59-a2ea-4417-9327-37cfb1b138e1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:57.263815 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648637c9-bb5f-41ba-bb04-4b079ce60388 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:57.264134 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648637c9-bb5f-41ba-bb04-4b079ce60388 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:57.264413 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648637c9-bb5f-41ba-bb04-4b079ce60388 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:57.265945 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 154/310] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:57.272504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb90ebb-e72a-4c6d-9606-345d32e310da None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:57.273115 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb90ebb-e72a-4c6d-9606-345d32e310da None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:57.273617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb90ebb-e72a-4c6d-9606-345d32e310da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:57.293071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633583b7-9531-44fb-b912-c86954b071fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:57.293750 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633583b7-9531-44fb-b912-c86954b071fd None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:57.294036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633583b7-9531-44fb-b912-c86954b071fd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:57.294317 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633583b7-9531-44fb-b912-c86954b071fd None admin] PATH_INFO: `/v3/system/users/6af8df8542514fd0aeb9b903b993bae5/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:57.296038 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633583b7-9531-44fb-b912-c86954b071fd None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6af8df8542514fd0aeb9b903b993bae5, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:57.297407 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-2bb90ebb-e72a-4c6d-9606-345d32e310da None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:57.297530 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633583b7-9531-44fb-b912-c86954b071fd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:57.298639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2bb90ebb-e72a-4c6d-9606-345d32e310da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:57.374734 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 155/311] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:57.392008 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 157/312] 10.222.0.49 () {58 vars in 1334 bytes} [Wed Aug 10 12:22:57 2022] PUT /identity/v3/system/users/6af8df8542514fd0aeb9b903b993bae5/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:22:57.397764 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb90ebb-e72a-4c6d-9606-345d32e310da None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:57.507734 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:57.515158 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb90ebb-e72a-4c6d-9606-345d32e310da None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:57.555863 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f34219-fa5b-42d2-be43-732f52df7d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:57.556524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f34219-fa5b-42d2-be43-732f52df7d7d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:57.556806 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f34219-fa5b-42d2-be43-732f52df7d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:57.557086 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f34219-fa5b-42d2-be43-732f52df7d7d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:57.569548 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f34219-fa5b-42d2-be43-732f52df7d7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:57.574592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-41f34219-fa5b-42d2-be43-732f52df7d7d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:22:57.574592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:57.574592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:57.574592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:57.574592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:57.574592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:57.574592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:57.574592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:57.574592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:57.574592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:57.574592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:57.574592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:57.574592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:57.574592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:57.574592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:57.574592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:57.575499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:57.575499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:57.575499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:57.575499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:57.575499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:57.575499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:57.575499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:57.575499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:57.575499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:57.575499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:57.575499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:57.575499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:57.575499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:57.575499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:57.575499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:57.575499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:57.576323 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:57.576323 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:57.576323 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:57.576323 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:57.576323 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:57.576323 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:57.576323 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:57.576323 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:57.576323 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:57.576323 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:57.576323 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:22:57.576323 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:22:57.576323 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 156/313] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:57.583430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633583b7-9531-44fb-b912-c86954b071fd None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:57.696598 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633583b7-9531-44fb-b912-c86954b071fd None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:57.734919 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5bc496-d259-4649-af88-45da1378235d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:57.735558 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bc496-d259-4649-af88-45da1378235d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:57.735839 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bc496-d259-4649-af88-45da1378235d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:57.736110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bc496-d259-4649-af88-45da1378235d None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:57.736777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5bc496-d259-4649-af88-45da1378235d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:57.737081 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5bc496-d259-4649-af88-45da1378235d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:57.738466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5bc496-d259-4649-af88-45da1378235d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:57.747542 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 158/314] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 12:22:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:57.753081 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f34219-fa5b-42d2-be43-732f52df7d7d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:57.866523 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f34219-fa5b-42d2-be43-732f52df7d7d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:57.903808 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380ebd4-6ccd-451b-845f-34ce92fb6e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:57.904866 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380ebd4-6ccd-451b-845f-34ce92fb6e1b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:57.905343 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380ebd4-6ccd-451b-845f-34ce92fb6e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:57.905842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380ebd4-6ccd-451b-845f-34ce92fb6e1b None admin] PATH_INFO: `/v3/users/67d9ac49973a425686988215f938c364` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:57.907243 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d380ebd4-6ccd-451b-845f-34ce92fb6e1b None admin] RBAC: Authorizing `identity:get_user(user_id=67d9ac49973a425686988215f938c364)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:57.908981 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d380ebd4-6ccd-451b-845f-34ce92fb6e1b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:57.911108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 157/315] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:57 2022] GET /identity/v3/users/67d9ac49973a425686988215f938c364 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:57.917298 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5bc496-d259-4649-af88-45da1378235d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:57.928339 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:58.031333 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5bc496-d259-4649-af88-45da1378235d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:58.066443 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ce9d3-474a-4af5-8816-f47dde372026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:58.067061 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ce9d3-474a-4af5-8816-f47dde372026 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:58.067370 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ce9d3-474a-4af5-8816-f47dde372026 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:58.067645 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ce9d3-474a-4af5-8816-f47dde372026 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:58.137058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ce9d3-474a-4af5-8816-f47dde372026 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:58.138515 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ce9d3-474a-4af5-8816-f47dde372026 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:58.155504 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:58.241081 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 159/316] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:22:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:22:58.246760 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380ebd4-6ccd-451b-845f-34ce92fb6e1b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:58.274780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:58.350735 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380ebd4-6ccd-451b-845f-34ce92fb6e1b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:58.387448 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5abc50-cd44-44fd-bb44-e89712ef6794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:58.388108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5abc50-cd44-44fd-bb44-e89712ef6794 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:58.388392 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5abc50-cd44-44fd-bb44-e89712ef6794 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:58.388652 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5abc50-cd44-44fd-bb44-e89712ef6794 None admin] PATH_INFO: `/v3/projects/b287589e923744ec9ed8d9c9f4196506` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:58.389867 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5abc50-cd44-44fd-bb44-e89712ef6794 None admin] RBAC: Authorizing `identity:get_project(project_id=b287589e923744ec9ed8d9c9f4196506)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:58.391380 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5abc50-cd44-44fd-bb44-e89712ef6794 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:58.393413 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 158/317] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:22:58 2022] GET /identity/v3/projects/b287589e923744ec9ed8d9c9f4196506 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:58.399944 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ce9d3-474a-4af5-8816-f47dde372026 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:58.505598 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ce9d3-474a-4af5-8816-f47dde372026 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:58.513575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:58.542027 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092832e-0354-4927-809e-06e56949ccc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:58.542642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092832e-0354-4927-809e-06e56949ccc7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:58.542926 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092832e-0354-4927-809e-06e56949ccc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:58.543216 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092832e-0354-4927-809e-06e56949ccc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:58.543950 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092832e-0354-4927-809e-06e56949ccc7 None admin] RBAC: Adding query filter params (role.id=78c5ea5c9f154c6da4912cfc338b6474, scope.project.id=b287589e923744ec9ed8d9c9f4196506, user.id=67d9ac49973a425686988215f938c364) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:58.544264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092832e-0354-4927-809e-06e56949ccc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:58.545708 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092832e-0354-4927-809e-06e56949ccc7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:58.552281 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 160/318] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:22:58 2022] GET /identity/v3/role_assignments?user.id=67d9ac49973a425686988215f938c364&scope.project.id=b287589e923744ec9ed8d9c9f4196506&role.id=78c5ea5c9f154c6da4912cfc338b6474 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:58.869518 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d5998-64bf-4cd9-82f6-88a7c66e9a1d None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:58.869796 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d5998-64bf-4cd9-82f6-88a7c66e9a1d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:58.870329 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d5998-64bf-4cd9-82f6-88a7c66e9a1d None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:58.871705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 159/319] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:58.876998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1afbd-d208-4224-b6a6-0bd52bb0b6c9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:58.877292 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1afbd-d208-4224-b6a6-0bd52bb0b6c9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:58.877562 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1afbd-d208-4224-b6a6-0bd52bb0b6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:58.895165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-60d1afbd-d208-4224-b6a6-0bd52bb0b6c9 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:58.896381 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-60d1afbd-d208-4224-b6a6-0bd52bb0b6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:58.962440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 161/320] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:22:58.985339 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d5998-64bf-4cd9-82f6-88a7c66e9a1d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:59.079276 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:59.090125 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d5998-64bf-4cd9-82f6-88a7c66e9a1d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:59.125936 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6514817-1e6b-4ca0-81ed-9e548df169cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:59.126500 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6514817-1e6b-4ca0-81ed-9e548df169cb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:59.126779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6514817-1e6b-4ca0-81ed-9e548df169cb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:59.127051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6514817-1e6b-4ca0-81ed-9e548df169cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:59.128186 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6514817-1e6b-4ca0-81ed-9e548df169cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:59.129713 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6514817-1e6b-4ca0-81ed-9e548df169cb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:59.136836 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 160/321] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:22:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:59.138445 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1afbd-d208-4224-b6a6-0bd52bb0b6c9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:59.238191 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:59.241895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1afbd-d208-4224-b6a6-0bd52bb0b6c9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:59.285803 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acaf69-c3cb-480c-a093-8492db1069a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:59.286442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acaf69-c3cb-480c-a093-8492db1069a0 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:59.286722 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acaf69-c3cb-480c-a093-8492db1069a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:59.287057 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acaf69-c3cb-480c-a093-8492db1069a0 None admin] PATH_INFO: `/v3/groups` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:59.287768 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acaf69-c3cb-480c-a093-8492db1069a0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:59.289249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acaf69-c3cb-480c-a093-8492db1069a0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:59.323350 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33898f2b-d8a5-4011-bcc9-6b255495a992 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:59.323850 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33898f2b-d8a5-4011-bcc9-6b255495a992 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:59.323933 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33898f2b-d8a5-4011-bcc9-6b255495a992 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:59.325227 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 161/322] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:22:59.330050 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64190350-5e66-426e-91f2-e53381b55dab None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:59.330335 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64190350-5e66-426e-91f2-e53381b55dab None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:59.330653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64190350-5e66-426e-91f2-e53381b55dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:59.348838 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-64190350-5e66-426e-91f2-e53381b55dab None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:59.350642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-64190350-5e66-426e-91f2-e53381b55dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:59.403511 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 162/323] 10.222.0.49 () {60 vars in 1134 bytes} [Wed Aug 10 12:22:59 2022] POST /identity/v3/groups => generated 226 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:22:59.422220 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 162/324] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:22:59.445947 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acaf69-c3cb-480c-a093-8492db1069a0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:59.541000 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30ae9f-0c07-4e7c-91cd-fb6701a1a972 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:59.541296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30ae9f-0c07-4e7c-91cd-fb6701a1a972 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:59.541556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30ae9f-0c07-4e7c-91cd-fb6701a1a972 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:59.542903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 163/325] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:22:59.548967 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02a5fc-12a7-4ace-a4cc-cc3afa9a7efc None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:59.549513 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02a5fc-12a7-4ace-a4cc-cc3afa9a7efc None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:59.550064 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02a5fc-12a7-4ace-a4cc-cc3afa9a7efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:59.556940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acaf69-c3cb-480c-a093-8492db1069a0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:59.570408 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-de02a5fc-12a7-4ace-a4cc-cc3afa9a7efc None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:59.571662 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-de02a5fc-12a7-4ace-a4cc-cc3afa9a7efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:59.596402 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c1ef8-7ebc-47b0-bc6a-bb8f28f3927c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:59.597045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c1ef8-7ebc-47b0-bc6a-bb8f28f3927c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:59.597326 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c1ef8-7ebc-47b0-bc6a-bb8f28f3927c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:59.597599 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c1ef8-7ebc-47b0-bc6a-bb8f28f3927c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:59.609068 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c1ef8-7ebc-47b0-bc6a-bb8f28f3927c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:59.614397 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-511c1ef8-7ebc-47b0-bc6a-bb8f28f3927c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:22:59.614397 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:59.614397 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:59.614397 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:59.614397 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:59.614397 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:59.614397 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:59.614397 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:59.614397 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:59.614397 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:59.614397 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:59.614397 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:59.614397 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:59.614397 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:59.614397 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:59.614397 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:59.615332 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:59.615332 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:59.615332 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:59.615332 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:59.615332 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:59.615332 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:59.615332 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:59.615332 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:59.615332 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:59.615332 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:59.615332 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:59.615332 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:59.615332 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:59.615332 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:59.615332 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:59.615332 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:59.616917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:59.616917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:59.616917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:59.616917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:59.616917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:59.616917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:59.616917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:59.616917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:59.616917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:59.616917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:59.616917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:22:59.616917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:22:59.616917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 163/326] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:59.622051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c1ef8-7ebc-47b0-bc6a-bb8f28f3927c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:59.649253 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:59.661033 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 164/327] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:22:59.683556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02a5fc-12a7-4ace-a4cc-cc3afa9a7efc None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:59.705472 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:59.740300 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c1ef8-7ebc-47b0-bc6a-bb8f28f3927c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:59.782194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3da67-075c-496c-8386-db71a4737e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:59.782819 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3da67-075c-496c-8386-db71a4737e3a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:59.783104 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3da67-075c-496c-8386-db71a4737e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:59.783455 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3da67-075c-496c-8386-db71a4737e3a None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:59.784133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e3da67-075c-496c-8386-db71a4737e3a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:59.784443 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e3da67-075c-496c-8386-db71a4737e3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:59.785843 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e3da67-075c-496c-8386-db71a4737e3a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:59.800267 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 164/328] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 12:22:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:59.806366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3da67-075c-496c-8386-db71a4737e3a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:59.815761 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02a5fc-12a7-4ace-a4cc-cc3afa9a7efc None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:59.861110 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed6227-cc54-44e8-8db7-7a64d32532ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:59.861872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed6227-cc54-44e8-8db7-7a64d32532ce None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:59.862153 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed6227-cc54-44e8-8db7-7a64d32532ce None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:59.862428 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed6227-cc54-44e8-8db7-7a64d32532ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:59.875688 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed6227-cc54-44e8-8db7-7a64d32532ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:59.881026 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-65ed6227-cc54-44e8-8db7-7a64d32532ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:22:59.881026 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:59.881026 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:59.881026 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:59.881026 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:59.881026 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:59.881026 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:59.881026 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:59.881026 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:59.881026 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:59.881026 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:59.881026 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:59.881026 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:59.881026 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:59.881026 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:59.881026 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:59.882443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:59.882443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:59.882443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:59.882443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:59.882443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:59.882443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:59.882443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:59.882443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:59.882443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:59.882443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:59.882443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:59.882443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:59.882443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:59.882443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:59.882443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:59.882443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:59.883251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:59.883251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:59.883251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:59.883251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:59.883251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:59.883251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:59.883251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:59.883251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:59.883251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:59.883251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:59.883251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:22:59.883251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:22:59.883251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 165/329] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:22:59.890212 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed6227-cc54-44e8-8db7-7a64d32532ce None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:59.905828 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:59.926659 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3da67-075c-496c-8386-db71a4737e3a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:59.965552 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145a3cf-1610-4274-9f78-cf12cd6d5c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:59.966201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145a3cf-1610-4274-9f78-cf12cd6d5c0d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:59.966478 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145a3cf-1610-4274-9f78-cf12cd6d5c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:59.966853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145a3cf-1610-4274-9f78-cf12cd6d5c0d None admin] PATH_INFO: `/v3/users/27dbe202509f4efe998a9b7b49028df3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:59.968013 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145a3cf-1610-4274-9f78-cf12cd6d5c0d None admin] RBAC: Authorizing `identity:get_user(user_id=27dbe202509f4efe998a9b7b49028df3)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:59.969660 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145a3cf-1610-4274-9f78-cf12cd6d5c0d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:59.971687 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 165/330] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:22:59 2022] GET /identity/v3/users/27dbe202509f4efe998a9b7b49028df3 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:59.982661 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145a3cf-1610-4274-9f78-cf12cd6d5c0d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:59.987674 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:00.026795 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed6227-cc54-44e8-8db7-7a64d32532ce None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:00.075662 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5549ff-a743-4f27-b56c-feff5aac7692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:00.076277 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5549ff-a743-4f27-b56c-feff5aac7692 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:00.076675 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5549ff-a743-4f27-b56c-feff5aac7692 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:00.076944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5549ff-a743-4f27-b56c-feff5aac7692 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:00.079494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5549ff-a743-4f27-b56c-feff5aac7692 None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:00.079820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5549ff-a743-4f27-b56c-feff5aac7692 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:00.081241 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5549ff-a743-4f27-b56c-feff5aac7692 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:00.093657 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 166/331] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 12:22:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:00.099713 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5549ff-a743-4f27-b56c-feff5aac7692 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:00.124712 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145a3cf-1610-4274-9f78-cf12cd6d5c0d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:00.163679 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c97e35-41ad-44aa-be6a-67ce24a385e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:00.165098 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c97e35-41ad-44aa-be6a-67ce24a385e9 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:00.165573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c97e35-41ad-44aa-be6a-67ce24a385e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:00.166188 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c97e35-41ad-44aa-be6a-67ce24a385e9 None admin] PATH_INFO: `/v3/system/users/27dbe202509f4efe998a9b7b49028df3/roles/78c5ea5c9f154c6da4912cfc338b6474` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:00.167984 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c97e35-41ad-44aa-be6a-67ce24a385e9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=27dbe202509f4efe998a9b7b49028df3, role_id=78c5ea5c9f154c6da4912cfc338b6474)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:00.169781 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c97e35-41ad-44aa-be6a-67ce24a385e9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:00.232057 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 166/332] 10.222.0.49 () {58 vars in 1334 bytes} [Wed Aug 10 12:22:59 2022] PUT /identity/v3/system/users/27dbe202509f4efe998a9b7b49028df3/roles/78c5ea5c9f154c6da4912cfc338b6474 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:23:00.232339 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5549ff-a743-4f27-b56c-feff5aac7692 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:00.284066 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0148fea8-2493-4343-a5cd-13fb27cf40d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:00.285056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148fea8-2493-4343-a5cd-13fb27cf40d6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:00.285576 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148fea8-2493-4343-a5cd-13fb27cf40d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:00.286040 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148fea8-2493-4343-a5cd-13fb27cf40d6 None admin] PATH_INFO: `/v3/users/6af8df8542514fd0aeb9b903b993bae5` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:00.288288 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0148fea8-2493-4343-a5cd-13fb27cf40d6 None admin] RBAC: Authorizing `identity:get_user(user_id=6af8df8542514fd0aeb9b903b993bae5)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:00.290035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0148fea8-2493-4343-a5cd-13fb27cf40d6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:00.292401 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 167/333] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:23:00 2022] GET /identity/v3/users/6af8df8542514fd0aeb9b903b993bae5 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:23:00.299989 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c97e35-41ad-44aa-be6a-67ce24a385e9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:00.321872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:00.424160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c97e35-41ad-44aa-be6a-67ce24a385e9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:00.465610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c61d4-dd71-40de-b368-e4f2f6b774a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:00.466294 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c61d4-dd71-40de-b368-e4f2f6b774a9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:00.466573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c61d4-dd71-40de-b368-e4f2f6b774a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:00.466848 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c61d4-dd71-40de-b368-e4f2f6b774a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:00.467692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c61d4-dd71-40de-b368-e4f2f6b774a9 None admin] RBAC: Adding query filter params (role.id=872ba0c4212a4684b539a68fc2c20b25, user.id=6af8df8542514fd0aeb9b903b993bae5, scope.system=all) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:00.467997 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c61d4-dd71-40de-b368-e4f2f6b774a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:00.469672 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c61d4-dd71-40de-b368-e4f2f6b774a9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:00.484619 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 167/334] 10.222.0.49 () {56 vars in 1308 bytes} [Wed Aug 10 12:23:00 2022] GET /identity/v3/role_assignments?user.id=6af8df8542514fd0aeb9b903b993bae5&scope.system=all&role.id=872ba0c4212a4684b539a68fc2c20b25 => generated 510 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:01.879519 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65088db5-ab73-4b51-8d77-dc2472f813da None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:01.910105 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65088db5-ab73-4b51-8d77-dc2472f813da None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:01.910105 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65088db5-ab73-4b51-8d77-dc2472f813da None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:01.910105 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 168/335] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:01.910366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ace894-0f91-4347-95ef-8f55c4b7e891 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:01.910366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ace894-0f91-4347-95ef-8f55c4b7e891 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:01.910366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ace894-0f91-4347-95ef-8f55c4b7e891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:01.911754 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-44ace894-0f91-4347-95ef-8f55c4b7e891 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:01.913142 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-44ace894-0f91-4347-95ef-8f55c4b7e891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:02.000474 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 168/336] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:23:02.035406 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178cc76-1285-4f3b-869c-723639cefdd1 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:02.035745 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178cc76-1285-4f3b-869c-723639cefdd1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:02.035953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178cc76-1285-4f3b-869c-723639cefdd1 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:02.037698 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 169/337] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:02.043334 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595595a-38d1-440a-a018-c5ca36212693 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:02.043498 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178cc76-1285-4f3b-869c-723639cefdd1 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:02.043605 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595595a-38d1-440a-a018-c5ca36212693 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:02.043891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595595a-38d1-440a-a018-c5ca36212693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:02.050494 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:02.059655 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:02.069322 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c595595a-38d1-440a-a018-c5ca36212693 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:02.070585 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c595595a-38d1-440a-a018-c5ca36212693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:02.184194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 169/338] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:02.228883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595595a-38d1-440a-a018-c5ca36212693 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:02.237018 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178cc76-1285-4f3b-869c-723639cefdd1 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:02.290112 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc769d-b830-4b99-a87a-8ada1322e48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:02.290761 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc769d-b830-4b99-a87a-8ada1322e48f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:02.291043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc769d-b830-4b99-a87a-8ada1322e48f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:02.291344 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc769d-b830-4b99-a87a-8ada1322e48f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:02.312862 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc769d-b830-4b99-a87a-8ada1322e48f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:02.318326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-94dc769d-b830-4b99-a87a-8ada1322e48f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:23:02.318326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:02.318326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:02.318326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:02.318326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:02.318326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:02.318326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:02.318326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:02.318326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:02.318326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:02.318326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:02.318326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:02.318326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:02.318326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:02.318326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:02.318326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:02.320310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:02.320310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:02.320310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:02.320310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:02.320310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:02.320310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:02.320310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:02.320310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:02.320310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:02.320310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:02.320310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:02.320310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:02.320310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:02.320310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:02.320310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:02.320310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:02.321131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:02.321131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:02.321131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:02.321131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:02.321131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:02.321131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:02.321131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:02.321131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:02.321131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:02.321131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:02.321131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:23:02.321131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:02.325017 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 170/339] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:23:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:02.326223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc769d-b830-4b99-a87a-8ada1322e48f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:02.439003 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:02.481814 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc769d-b830-4b99-a87a-8ada1322e48f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:02.506642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595595a-38d1-440a-a018-c5ca36212693 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:02.527000 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c4b3b-cd05-447d-8683-e12655407be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:02.527996 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c4b3b-cd05-447d-8683-e12655407be1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:02.528437 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c4b3b-cd05-447d-8683-e12655407be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:02.528865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c4b3b-cd05-447d-8683-e12655407be1 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:02.529689 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c4b3b-cd05-447d-8683-e12655407be1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:02.530149 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c4b3b-cd05-447d-8683-e12655407be1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:02.531706 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c4b3b-cd05-447d-8683-e12655407be1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:02.541427 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 171/340] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 12:23:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:23:02.547026 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c4b3b-cd05-447d-8683-e12655407be1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:02.572213 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:02.582626 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bbdd08-dc91-4587-9ea2-bb898709097d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:02.590876 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbdd08-dc91-4587-9ea2-bb898709097d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:02.590876 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbdd08-dc91-4587-9ea2-bb898709097d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:02.590876 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbdd08-dc91-4587-9ea2-bb898709097d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:02.612372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bbdd08-dc91-4587-9ea2-bb898709097d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:02.624035 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-03bbdd08-dc91-4587-9ea2-bb898709097d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:23:02.624035 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:02.624035 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:02.624035 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:02.624035 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:02.624035 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:02.624035 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:02.624035 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:02.624035 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:02.624035 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:02.624035 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:02.624035 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:02.624035 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:02.624035 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:02.624035 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:02.624035 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:02.625170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:02.625170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:02.625170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:02.625170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:02.625170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:02.625170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:02.625170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:02.625170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:02.625170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:02.625170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:02.625170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:02.625170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:02.625170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:02.625170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:02.625170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:02.625170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:02.626325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:02.626325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:02.626325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:02.626325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:02.626325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:02.626325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:02.626325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:02.626325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:02.626325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:02.626325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:02.626325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:23:02.626325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:02.626325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 170/341] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 12:23:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:23:02.632156 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bbdd08-dc91-4587-9ea2-bb898709097d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:02.754947 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c4b3b-cd05-447d-8683-e12655407be1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:02.814703 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:02.831013 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69d60d-88e1-421e-af44-b98fe47f66a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:02.832091 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69d60d-88e1-421e-af44-b98fe47f66a1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:02.832528 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69d60d-88e1-421e-af44-b98fe47f66a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:02.833002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69d60d-88e1-421e-af44-b98fe47f66a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:02.871538 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bbdd08-dc91-4587-9ea2-bb898709097d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:02.946782 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32457b-c4d2-46a2-aa0a-d48d9cb58118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:02.948099 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32457b-c4d2-46a2-aa0a-d48d9cb58118 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:02.948544 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32457b-c4d2-46a2-aa0a-d48d9cb58118 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:02.949012 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32457b-c4d2-46a2-aa0a-d48d9cb58118 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:02.949885 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd32457b-c4d2-46a2-aa0a-d48d9cb58118 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:02.950502 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd32457b-c4d2-46a2-aa0a-d48d9cb58118 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:02.952490 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd32457b-c4d2-46a2-aa0a-d48d9cb58118 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:02.965936 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 171/342] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:02.973139 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32457b-c4d2-46a2-aa0a-d48d9cb58118 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:02.987150 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc69d60d-88e1-421e-af44-b98fe47f66a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:02.989282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc69d60d-88e1-421e-af44-b98fe47f66a1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:03.099296 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:03.186621 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 172/343] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:23:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:23:03.192407 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69d60d-88e1-421e-af44-b98fe47f66a1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:03.203098 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32457b-c4d2-46a2-aa0a-d48d9cb58118 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:03.243679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:03.288502 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7aa4f-786e-4dcb-9b21-52026d4e976e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:03.289963 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7aa4f-786e-4dcb-9b21-52026d4e976e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:03.290765 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7aa4f-786e-4dcb-9b21-52026d4e976e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:03.291279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7aa4f-786e-4dcb-9b21-52026d4e976e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:03.385052 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f7aa4f-786e-4dcb-9b21-52026d4e976e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:03.387056 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f7aa4f-786e-4dcb-9b21-52026d4e976e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:03.406342 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69d60d-88e1-421e-af44-b98fe47f66a1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:03.451624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:03.472372 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5dcc70-7f16-4624-94bf-f23e8353a705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:03.472602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5dcc70-7f16-4624-94bf-f23e8353a705 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:03.472855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5dcc70-7f16-4624-94bf-f23e8353a705 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:03.473128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5dcc70-7f16-4624-94bf-f23e8353a705 None admin] PATH_INFO: `/v3/projects/9c9a0023662b41afa1d7bec05fd12eda` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:03.474344 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5dcc70-7f16-4624-94bf-f23e8353a705 None admin] RBAC: Authorizing `identity:get_project(project_id=9c9a0023662b41afa1d7bec05fd12eda)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:03.476499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5dcc70-7f16-4624-94bf-f23e8353a705 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:03.478490 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 173/344] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:23:03 2022] GET /identity/v3/projects/9c9a0023662b41afa1d7bec05fd12eda => generated 298 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:23:03.479689 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7ed9e-d4d6-4827-81d2-0c85aa31b2c2 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:03.488724 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7ed9e-d4d6-4827-81d2-0c85aa31b2c2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:03.488724 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7ed9e-d4d6-4827-81d2-0c85aa31b2c2 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:03.488724 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 174/345] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:03.488724 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7ed9e-d4d6-4827-81d2-0c85aa31b2c2 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:03.501899 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 172/346] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:23:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:23:03.503680 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be7590-7bcc-4dc8-a3be-cec1df067b0e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:03.504275 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be7590-7bcc-4dc8-a3be-cec1df067b0e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:03.504738 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be7590-7bcc-4dc8-a3be-cec1df067b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:03.543656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-31be7590-7bcc-4dc8-a3be-cec1df067b0e None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:03.545887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-31be7590-7bcc-4dc8-a3be-cec1df067b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:03.645485 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:03.651551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 173/347] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:03.653026 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be7590-7bcc-4dc8-a3be-cec1df067b0e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:03.655833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7ed9e-d4d6-4827-81d2-0c85aa31b2c2 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:03.703602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f534916-be4b-4297-ab0c-7af5ce9b6668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:03.704241 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f534916-be4b-4297-ab0c-7af5ce9b6668 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:03.704519 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f534916-be4b-4297-ab0c-7af5ce9b6668 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:03.704792 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f534916-be4b-4297-ab0c-7af5ce9b6668 None admin] PATH_INFO: `/v3/groups/2cfd468a091f4dbba6eaa9d630e9ec6d` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:03.711556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f534916-be4b-4297-ab0c-7af5ce9b6668 None admin] RBAC: Authorizing `identity:get_group(group_id=2cfd468a091f4dbba6eaa9d630e9ec6d)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:03.713294 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f534916-be4b-4297-ab0c-7af5ce9b6668 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:03.715121 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 175/348] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 12:23:03 2022] GET /identity/v3/groups/2cfd468a091f4dbba6eaa9d630e9ec6d => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:03.717550 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45378c14-ad1e-4955-9028-99d6c8ff5fcd None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:03.717550 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45378c14-ad1e-4955-9028-99d6c8ff5fcd None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:03.717550 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45378c14-ad1e-4955-9028-99d6c8ff5fcd None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:03.718162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 176/349] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:03.719127 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe96cb5-a460-44b4-b776-569fccc35bc7 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:03.723466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe96cb5-a460-44b4-b776-569fccc35bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:03.724138 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe96cb5-a460-44b4-b776-569fccc35bc7 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:03.726425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 177/350] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:03.727167 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe96cb5-a460-44b4-b776-569fccc35bc7 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:03.749510 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:03.773555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be7590-7bcc-4dc8-a3be-cec1df067b0e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:03.826251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8f81c-4295-48e4-ac79-6810b4c97015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:03.827232 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8f81c-4295-48e4-ac79-6810b4c97015 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:03.827525 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8f81c-4295-48e4-ac79-6810b4c97015 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:03.827800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8f81c-4295-48e4-ac79-6810b4c97015 None admin] PATH_INFO: `/v3/projects/9c9a0023662b41afa1d7bec05fd12eda` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:03.829220 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8f81c-4295-48e4-ac79-6810b4c97015 None admin] RBAC: Authorizing `identity:get_project(project_id=9c9a0023662b41afa1d7bec05fd12eda)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:03.830837 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8f81c-4295-48e4-ac79-6810b4c97015 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:03.833093 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 174/351] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:23:03 2022] GET /identity/v3/projects/9c9a0023662b41afa1d7bec05fd12eda => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:03.838775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d6238-6a78-440e-aba1-57b0aa25f47d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:03.839062 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d6238-6a78-440e-aba1-57b0aa25f47d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:03.839361 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d6238-6a78-440e-aba1-57b0aa25f47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:03.859315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe96cb5-a460-44b4-b776-569fccc35bc7 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:03.861061 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-861d6238-6a78-440e-aba1-57b0aa25f47d None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:03.862253 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-861d6238-6a78-440e-aba1-57b0aa25f47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:03.914362 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5fbe8-c0db-453d-9ecd-5c25ee97c692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:03.915335 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5fbe8-c0db-453d-9ecd-5c25ee97c692 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:03.915815 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5fbe8-c0db-453d-9ecd-5c25ee97c692 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:03.916298 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5fbe8-c0db-453d-9ecd-5c25ee97c692 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:03.928873 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5fbe8-c0db-453d-9ecd-5c25ee97c692 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:03.934415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-16c5fbe8-c0db-453d-9ecd-5c25ee97c692 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:23:03.934415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:03.934415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:03.934415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:03.934415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:03.934415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:03.934415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:03.934415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:03.934415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:03.934415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:03.934415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:03.934415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:03.934415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:03.934415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:03.934415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:03.934415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:03.935326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:03.935326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:03.935326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:03.935326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:03.935326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:03.935326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:03.935326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:03.935326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:03.935326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:03.935326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:03.935326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:03.935326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:03.935326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:03.935326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:03.935326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:03.935326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:03.936145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:03.936145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:03.936145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:03.936145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:03.936145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:03.936145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:03.936145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:03.936145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:03.936145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:03.936145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:03.936145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:23:03.936145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:03.938369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 178/352] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:23:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:03.939356 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5fbe8-c0db-453d-9ecd-5c25ee97c692 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:03.946120 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 175/353] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:03.947829 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9713a5-274c-494c-a84f-8f7c0a19a0ea None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:03.948851 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9713a5-274c-494c-a84f-8f7c0a19a0ea None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:03.949291 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9713a5-274c-494c-a84f-8f7c0a19a0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:03.969021 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:03.973720 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5c9713a5-274c-494c-a84f-8f7c0a19a0ea None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:03.976081 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5c9713a5-274c-494c-a84f-8f7c0a19a0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:04.071263 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 176/354] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:04.071263 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9713a5-274c-494c-a84f-8f7c0a19a0ea None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:04.086799 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5fbe8-c0db-453d-9ecd-5c25ee97c692 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:04.092056 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:04.135280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2021a0-272e-46ea-95ea-a8b93bd8f4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:04.136245 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2021a0-272e-46ea-95ea-a8b93bd8f4ce None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:04.136729 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2021a0-272e-46ea-95ea-a8b93bd8f4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:04.137229 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2021a0-272e-46ea-95ea-a8b93bd8f4ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:04.138171 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2021a0-272e-46ea-95ea-a8b93bd8f4ce None admin] RBAC: Adding query filter params (group.id=2cfd468a091f4dbba6eaa9d630e9ec6d, role.id=872ba0c4212a4684b539a68fc2c20b25, scope.project.id=9c9a0023662b41afa1d7bec05fd12eda) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:04.138666 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2021a0-272e-46ea-95ea-a8b93bd8f4ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:04.140393 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2021a0-272e-46ea-95ea-a8b93bd8f4ce None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:04.148960 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 179/355] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 10 12:23:03 2022] GET /identity/v3/role_assignments?group.id=2cfd468a091f4dbba6eaa9d630e9ec6d&scope.project.id=9c9a0023662b41afa1d7bec05fd12eda&role.id=872ba0c4212a4684b539a68fc2c20b25 => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:04.155257 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2021a0-272e-46ea-95ea-a8b93bd8f4ce None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:04.206425 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9713a5-274c-494c-a84f-8f7c0a19a0ea None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:04.249166 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57088349-2f3d-4447-a9ae-c4db6b08bc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:04.249816 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57088349-2f3d-4447-a9ae-c4db6b08bc9e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:04.250097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57088349-2f3d-4447-a9ae-c4db6b08bc9e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:04.250370 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57088349-2f3d-4447-a9ae-c4db6b08bc9e None admin] PATH_INFO: `/v3/groups/2cfd468a091f4dbba6eaa9d630e9ec6d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:04.251656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57088349-2f3d-4447-a9ae-c4db6b08bc9e None admin] RBAC: Authorizing `identity:get_group(group_id=2cfd468a091f4dbba6eaa9d630e9ec6d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:04.253314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57088349-2f3d-4447-a9ae-c4db6b08bc9e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:04.254965 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 177/356] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 12:23:04 2022] GET /identity/v3/groups/2cfd468a091f4dbba6eaa9d630e9ec6d => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:04.261392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57088349-2f3d-4447-a9ae-c4db6b08bc9e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:04.273635 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:04.293984 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2021a0-272e-46ea-95ea-a8b93bd8f4ce None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:04.341607 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349b0a6-58c4-422a-b064-8476dc6dfa05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:04.342213 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349b0a6-58c4-422a-b064-8476dc6dfa05 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:04.342496 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349b0a6-58c4-422a-b064-8476dc6dfa05 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:04.342764 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349b0a6-58c4-422a-b064-8476dc6dfa05 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:04.343448 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a349b0a6-58c4-422a-b064-8476dc6dfa05 None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:04.343759 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a349b0a6-58c4-422a-b064-8476dc6dfa05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:04.345119 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a349b0a6-58c4-422a-b064-8476dc6dfa05 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:04.359197 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 180/357] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 12:23:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:04.359197 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349b0a6-58c4-422a-b064-8476dc6dfa05 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:04.385648 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:04.399539 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57088349-2f3d-4447-a9ae-c4db6b08bc9e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:04.439712 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639821b-ff51-4d66-be77-35cd2873e1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:04.440698 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639821b-ff51-4d66-be77-35cd2873e1e0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:04.441155 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639821b-ff51-4d66-be77-35cd2873e1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:04.441644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639821b-ff51-4d66-be77-35cd2873e1e0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:04.455870 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639821b-ff51-4d66-be77-35cd2873e1e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:04.461400 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-3639821b-ff51-4d66-be77-35cd2873e1e0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:23:04.461400 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:04.461400 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:04.461400 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:04.461400 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:04.461400 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:04.461400 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:04.461400 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:04.461400 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:04.461400 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:04.461400 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:04.461400 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:04.461400 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:04.461400 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:04.461400 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:04.461400 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:04.462305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:04.462305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:04.462305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:04.462305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:04.462305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:04.462305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:04.462305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:04.462305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:04.462305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:04.462305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:04.462305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:04.462305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:04.462305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:04.462305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:04.462305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:04.462305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:04.463111 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:04.463111 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:04.463111 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:04.463111 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:04.463111 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:04.463111 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:04.463111 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:04.463111 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:04.463111 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:04.463111 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:04.463111 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:23:04.463111 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:04.463861 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349b0a6-58c4-422a-b064-8476dc6dfa05 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:04.472161 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 178/358] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:23:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:04.472161 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639821b-ff51-4d66-be77-35cd2873e1e0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:04.510705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f20ad6-d38c-40b2-b9fb-2adb2e0f03cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:04.511395 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f20ad6-d38c-40b2-b9fb-2adb2e0f03cd None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:04.511672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f20ad6-d38c-40b2-b9fb-2adb2e0f03cd None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:04.511938 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f20ad6-d38c-40b2-b9fb-2adb2e0f03cd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:04.524722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f20ad6-d38c-40b2-b9fb-2adb2e0f03cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:04.530173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-a6f20ad6-d38c-40b2-b9fb-2adb2e0f03cd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:23:04.530173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:04.530173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:04.530173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:04.530173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:04.530173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:04.530173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:04.530173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:04.530173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:04.530173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:04.530173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:04.530173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:04.530173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:04.530173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:04.530173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:04.530173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:04.531062 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:04.531062 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:04.531062 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:04.531062 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:04.531062 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:04.531062 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:04.531062 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:04.531062 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:04.531062 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:04.531062 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:04.531062 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:04.531062 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:04.531062 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:04.531062 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:04.531062 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:04.531062 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:04.532010 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:04.532010 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:04.532010 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:04.532010 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:04.532010 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:04.532010 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:04.532010 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:04.532010 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:04.532010 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:04.532010 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:04.532010 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:23:04.532010 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:04.532010 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 181/359] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 12:23:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:04.532711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f20ad6-d38c-40b2-b9fb-2adb2e0f03cd None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:04.548224 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:04.607037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639821b-ff51-4d66-be77-35cd2873e1e0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:04.650190 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d6896-383a-4348-a8e6-b07736cb6e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:04.650804 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d6896-383a-4348-a8e6-b07736cb6e14 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:04.651081 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d6896-383a-4348-a8e6-b07736cb6e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:04.651382 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d6896-383a-4348-a8e6-b07736cb6e14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:04.652107 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63d6896-383a-4348-a8e6-b07736cb6e14 None admin] RBAC: Adding query filter params (group.id=2cfd468a091f4dbba6eaa9d630e9ec6d, role.id=0f58d2782a2840648bbbc2ab26ec02df, scope.project.id=9c9a0023662b41afa1d7bec05fd12eda) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:04.652412 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63d6896-383a-4348-a8e6-b07736cb6e14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:04.653849 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63d6896-383a-4348-a8e6-b07736cb6e14 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:04.662202 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 179/360] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 10 12:23:04 2022] GET /identity/v3/role_assignments?group.id=2cfd468a091f4dbba6eaa9d630e9ec6d&scope.project.id=9c9a0023662b41afa1d7bec05fd12eda&role.id=0f58d2782a2840648bbbc2ab26ec02df => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:04.662202 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca524b2-9e85-40ef-b6b8-71db2c81b820 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:04.662202 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca524b2-9e85-40ef-b6b8-71db2c81b820 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:04.663087 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca524b2-9e85-40ef-b6b8-71db2c81b820 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:04.665341 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 180/361] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:04.667217 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca524b2-9e85-40ef-b6b8-71db2c81b820 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:04.670969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:04.686778 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f20ad6-d38c-40b2-b9fb-2adb2e0f03cd None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:04.727273 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4c170-939b-4f27-80a1-9d51fb105788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:04.727880 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4c170-939b-4f27-80a1-9d51fb105788 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:04.728160 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4c170-939b-4f27-80a1-9d51fb105788 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:04.728486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4c170-939b-4f27-80a1-9d51fb105788 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:04.802003 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca524b2-9e85-40ef-b6b8-71db2c81b820 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:04.833084 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4c170-939b-4f27-80a1-9d51fb105788 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:04.836155 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4c170-939b-4f27-80a1-9d51fb105788 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:04.847302 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06d3fa-6899-46ea-a41d-9c6e1850859e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:04.847925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06d3fa-6899-46ea-a41d-9c6e1850859e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:04.848207 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06d3fa-6899-46ea-a41d-9c6e1850859e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:04.848472 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06d3fa-6899-46ea-a41d-9c6e1850859e None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:04.849138 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb06d3fa-6899-46ea-a41d-9c6e1850859e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:04.849442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb06d3fa-6899-46ea-a41d-9c6e1850859e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:04.850751 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:04.851650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb06d3fa-6899-46ea-a41d-9c6e1850859e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:04.866150 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 181/362] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 12:23:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:04.867006 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06d3fa-6899-46ea-a41d-9c6e1850859e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:04.958259 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:04.995604 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06d3fa-6899-46ea-a41d-9c6e1850859e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:05.013300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 182/363] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:23:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:23:05.015817 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d217e28-1439-4ebb-aae9-9e83f0a01422 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:05.016104 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d217e28-1439-4ebb-aae9-9e83f0a01422 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:05.016380 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d217e28-1439-4ebb-aae9-9e83f0a01422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:05.034849 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b323e2b7-bf9d-4549-958c-9102d7e4796f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:05.035467 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323e2b7-bf9d-4549-958c-9102d7e4796f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:05.035746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323e2b7-bf9d-4549-958c-9102d7e4796f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:05.036017 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323e2b7-bf9d-4549-958c-9102d7e4796f None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:05.036680 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b323e2b7-bf9d-4549-958c-9102d7e4796f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:05.036982 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b323e2b7-bf9d-4549-958c-9102d7e4796f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:05.038950 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b323e2b7-bf9d-4549-958c-9102d7e4796f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:05.047807 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 182/364] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:05.048640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b323e2b7-bf9d-4549-958c-9102d7e4796f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:05.053128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6d217e28-1439-4ebb-aae9-9e83f0a01422 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:05.054659 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6d217e28-1439-4ebb-aae9-9e83f0a01422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:05.160517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 183/365] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:05.162840 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d217e28-1439-4ebb-aae9-9e83f0a01422 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:05.165316 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b323e2b7-bf9d-4549-958c-9102d7e4796f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:05.199942 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:05.208500 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2fd50-e743-4af6-a077-7893c54a2615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:05.215414 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2fd50-e743-4af6-a077-7893c54a2615 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:05.215414 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2fd50-e743-4af6-a077-7893c54a2615 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:05.215414 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2fd50-e743-4af6-a077-7893c54a2615 None admin] PATH_INFO: `/v3/users/27dbe202509f4efe998a9b7b49028df3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:05.215414 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2fd50-e743-4af6-a077-7893c54a2615 None admin] RBAC: Authorizing `identity:get_user(user_id=27dbe202509f4efe998a9b7b49028df3)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:05.215414 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2fd50-e743-4af6-a077-7893c54a2615 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:05.215788 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:05.216061 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 183/366] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:23:05 2022] GET /identity/v3/users/27dbe202509f4efe998a9b7b49028df3 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:05.217114 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2fd50-e743-4af6-a077-7893c54a2615 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:05.295290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d217e28-1439-4ebb-aae9-9e83f0a01422 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:05.335497 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93e13d-efca-4e12-b4ff-aee2cfef9892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:05.336203 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93e13d-efca-4e12-b4ff-aee2cfef9892 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:05.336484 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93e13d-efca-4e12-b4ff-aee2cfef9892 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:05.336757 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93e13d-efca-4e12-b4ff-aee2cfef9892 None admin] PATH_INFO: `/v3/projects/b287589e923744ec9ed8d9c9f4196506` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:05.337932 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c93e13d-efca-4e12-b4ff-aee2cfef9892 None admin] RBAC: Authorizing `identity:get_project(project_id=b287589e923744ec9ed8d9c9f4196506)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:05.339405 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c93e13d-efca-4e12-b4ff-aee2cfef9892 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:05.341033 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 184/367] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:23:05 2022] GET /identity/v3/projects/b287589e923744ec9ed8d9c9f4196506 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:05.341779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93e13d-efca-4e12-b4ff-aee2cfef9892 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:05.343727 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2fd50-e743-4af6-a077-7893c54a2615 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:05.388429 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81a57b-eb95-4c41-a24d-41a72e6589a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:05.389326 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81a57b-eb95-4c41-a24d-41a72e6589a2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:05.389621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81a57b-eb95-4c41-a24d-41a72e6589a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:05.389896 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81a57b-eb95-4c41-a24d-41a72e6589a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:05.456042 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93e13d-efca-4e12-b4ff-aee2cfef9892 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:05.468060 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:05.476559 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:05.484852 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81a57b-eb95-4c41-a24d-41a72e6589a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:05.486756 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81a57b-eb95-4c41-a24d-41a72e6589a2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:05.503391 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2463ca-649d-4bcf-93d0-8e9dd5601c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:05.504909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2463ca-649d-4bcf-93d0-8e9dd5601c5a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:05.506490 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2463ca-649d-4bcf-93d0-8e9dd5601c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:05.506770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2463ca-649d-4bcf-93d0-8e9dd5601c5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:05.521070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2463ca-649d-4bcf-93d0-8e9dd5601c5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:05.527260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-5d2463ca-649d-4bcf-93d0-8e9dd5601c5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:23:05.527260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:05.527260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:05.527260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:05.527260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:05.527260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:05.527260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:05.527260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:05.527260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:05.527260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:05.527260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:05.527260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:05.527260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:05.527260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:05.527260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:05.527260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:05.528145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:05.528145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:05.528145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:05.528145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:05.528145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:05.528145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:05.528145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:05.528145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:05.528145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:05.528145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:05.528145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:05.528145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:05.528145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:05.528145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:05.528145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:05.528145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:05.528944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:05.528944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:05.528944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:05.528944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:05.528944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:05.528944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:05.528944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:05.528944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:05.528944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:05.528944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:05.528944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:23:05.528944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:05.530277 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 185/368] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:23:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:05.530430 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2463ca-649d-4bcf-93d0-8e9dd5601c5a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:05.610232 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 184/369] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:23:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:23:05.611086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81a57b-eb95-4c41-a24d-41a72e6589a2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:05.657992 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2463ca-649d-4bcf-93d0-8e9dd5601c5a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:05.695694 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b1e8e-7928-4b4e-93db-c161484eb4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:05.696304 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b1e8e-7928-4b4e-93db-c161484eb4f8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:05.696587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b1e8e-7928-4b4e-93db-c161484eb4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:05.696856 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b1e8e-7928-4b4e-93db-c161484eb4f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:05.697573 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b1e8e-7928-4b4e-93db-c161484eb4f8 None admin] RBAC: Adding query filter params (role.id=78c5ea5c9f154c6da4912cfc338b6474, user.id=27dbe202509f4efe998a9b7b49028df3, scope.system=all) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:05.697883 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b1e8e-7928-4b4e-93db-c161484eb4f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:05.699860 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b1e8e-7928-4b4e-93db-c161484eb4f8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:05.712987 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 186/370] 10.222.0.49 () {56 vars in 1308 bytes} [Wed Aug 10 12:23:05 2022] GET /identity/v3/role_assignments?user.id=27dbe202509f4efe998a9b7b49028df3&scope.system=all&role.id=78c5ea5c9f154c6da4912cfc338b6474 => generated 510 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:05.713810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b1e8e-7928-4b4e-93db-c161484eb4f8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:05.744356 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:05.745000 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:05.751824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81a57b-eb95-4c41-a24d-41a72e6589a2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:05.808743 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2a0c0-df94-4bf3-b2c1-08f4d593e8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:05.809319 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2a0c0-df94-4bf3-b2c1-08f4d593e8bc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:05.809598 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2a0c0-df94-4bf3-b2c1-08f4d593e8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:05.809866 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2a0c0-df94-4bf3-b2c1-08f4d593e8bc None admin] PATH_INFO: `/v3/groups/2cfd468a091f4dbba6eaa9d630e9ec6d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:05.811022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c2a0c0-df94-4bf3-b2c1-08f4d593e8bc None admin] RBAC: Authorizing `identity:get_group(group_id=2cfd468a091f4dbba6eaa9d630e9ec6d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:05.812528 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c2a0c0-df94-4bf3-b2c1-08f4d593e8bc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:05.817327 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 185/371] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 12:23:05 2022] GET /identity/v3/groups/2cfd468a091f4dbba6eaa9d630e9ec6d => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:05.818261 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2a0c0-df94-4bf3-b2c1-08f4d593e8bc None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:05.833211 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b1e8e-7928-4b4e-93db-c161484eb4f8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:05.873522 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b09fb6-6aea-41ec-9b6c-895078f490b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:05.874145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b09fb6-6aea-41ec-9b6c-895078f490b5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:05.874424 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b09fb6-6aea-41ec-9b6c-895078f490b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:05.874700 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b09fb6-6aea-41ec-9b6c-895078f490b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:05.875421 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b09fb6-6aea-41ec-9b6c-895078f490b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:05.875736 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b09fb6-6aea-41ec-9b6c-895078f490b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:05.877118 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b09fb6-6aea-41ec-9b6c-895078f490b5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:05.887817 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 187/372] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:23:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:05.888684 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b09fb6-6aea-41ec-9b6c-895078f490b5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:05.984661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:06.002363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b09fb6-6aea-41ec-9b6c-895078f490b5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:06.005349 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2a0c0-df94-4bf3-b2c1-08f4d593e8bc None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:06.042220 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f9d8a-c05d-4144-a550-8e9c7d42ecef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:06.042616 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f9d8a-c05d-4144-a550-8e9c7d42ecef None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:06.042894 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f9d8a-c05d-4144-a550-8e9c7d42ecef None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:06.043167 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f9d8a-c05d-4144-a550-8e9c7d42ecef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:06.043997 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f9d8a-c05d-4144-a550-8e9c7d42ecef None admin] RBAC: Adding query filter params (group.id=2cfd468a091f4dbba6eaa9d630e9ec6d, role.id=872ba0c4212a4684b539a68fc2c20b25, scope.project.id=b287589e923744ec9ed8d9c9f4196506) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:06.044312 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f9d8a-c05d-4144-a550-8e9c7d42ecef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:06.045960 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f9d8a-c05d-4144-a550-8e9c7d42ecef None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:06.052219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce2986-2cc9-4309-9149-e106ad3b8054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:06.052786 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 188/373] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 10 12:23:05 2022] GET /identity/v3/role_assignments?group.id=2cfd468a091f4dbba6eaa9d630e9ec6d&scope.project.id=b287589e923744ec9ed8d9c9f4196506&role.id=872ba0c4212a4684b539a68fc2c20b25 => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:06.054285 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f9d8a-c05d-4144-a550-8e9c7d42ecef None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:06.054581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce2986-2cc9-4309-9149-e106ad3b8054 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:06.055152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce2986-2cc9-4309-9149-e106ad3b8054 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:06.055648 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce2986-2cc9-4309-9149-e106ad3b8054 None admin] PATH_INFO: `/v3/projects/b287589e923744ec9ed8d9c9f4196506` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:06.057229 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce2986-2cc9-4309-9149-e106ad3b8054 None admin] RBAC: Authorizing `identity:get_project(project_id=b287589e923744ec9ed8d9c9f4196506)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:06.059161 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce2986-2cc9-4309-9149-e106ad3b8054 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:06.061572 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 186/374] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:23:05 2022] GET /identity/v3/projects/b287589e923744ec9ed8d9c9f4196506 => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:06.072496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce2986-2cc9-4309-9149-e106ad3b8054 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:06.133231 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:06.172470 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f9d8a-c05d-4144-a550-8e9c7d42ecef None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:06.211060 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79d8732-ae3b-46b6-83cc-63678a224e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:06.211813 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79d8732-ae3b-46b6-83cc-63678a224e81 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:06.212093 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79d8732-ae3b-46b6-83cc-63678a224e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:06.212365 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79d8732-ae3b-46b6-83cc-63678a224e81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:06.234562 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:06.291047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79d8732-ae3b-46b6-83cc-63678a224e81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:06.293622 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79d8732-ae3b-46b6-83cc-63678a224e81 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:06.297736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce2986-2cc9-4309-9149-e106ad3b8054 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:06.392554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ede32-744c-45d7-93c9-0dbb1fee01fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:06.393279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ede32-744c-45d7-93c9-0dbb1fee01fe None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:06.393665 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ede32-744c-45d7-93c9-0dbb1fee01fe None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:06.394043 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ede32-744c-45d7-93c9-0dbb1fee01fe None admin] PATH_INFO: `/v3/groups/2cfd468a091f4dbba6eaa9d630e9ec6d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:06.395477 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ede32-744c-45d7-93c9-0dbb1fee01fe None admin] RBAC: Authorizing `identity:get_group(group_id=2cfd468a091f4dbba6eaa9d630e9ec6d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:06.397149 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ede32-744c-45d7-93c9-0dbb1fee01fe None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:06.399134 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 189/376] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:23:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:23:06.399287 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 187/376] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 12:23:06 2022] GET /identity/v3/groups/2cfd468a091f4dbba6eaa9d630e9ec6d => generated 226 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:06.404673 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ede32-744c-45d7-93c9-0dbb1fee01fe None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:06.408035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79d8732-ae3b-46b6-83cc-63678a224e81 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:06.496231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:06.553805 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79d8732-ae3b-46b6-83cc-63678a224e81 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:06.578056 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:06.585887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ede32-744c-45d7-93c9-0dbb1fee01fe None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:06.590341 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156249d5-1dec-4579-828e-30265b53c527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:06.591281 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156249d5-1dec-4579-828e-30265b53c527 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:06.591739 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156249d5-1dec-4579-828e-30265b53c527 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:06.592186 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156249d5-1dec-4579-828e-30265b53c527 None admin] PATH_INFO: `/v3/projects/3c65e78604254a1dbcd377669ee26423` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:06.593522 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156249d5-1dec-4579-828e-30265b53c527 None admin] RBAC: Authorizing `identity:get_project(project_id=3c65e78604254a1dbcd377669ee26423)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:06.595252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156249d5-1dec-4579-828e-30265b53c527 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:06.597410 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 190/377] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:23:06 2022] GET /identity/v3/projects/3c65e78604254a1dbcd377669ee26423 => generated 344 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:06.603156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156249d5-1dec-4579-828e-30265b53c527 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:06.654294 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f48e65-acdd-4368-a72b-c2f0dc4e894a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:06.654603 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f48e65-acdd-4368-a72b-c2f0dc4e894a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:06.655558 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f48e65-acdd-4368-a72b-c2f0dc4e894a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:06.655558 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f48e65-acdd-4368-a72b-c2f0dc4e894a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:06.655890 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f48e65-acdd-4368-a72b-c2f0dc4e894a None admin] RBAC: Adding query filter params (group.id=2cfd468a091f4dbba6eaa9d630e9ec6d, role.id=0f58d2782a2840648bbbc2ab26ec02df, scope.project.id=b287589e923744ec9ed8d9c9f4196506) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:06.656206 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f48e65-acdd-4368-a72b-c2f0dc4e894a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:06.657750 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f48e65-acdd-4368-a72b-c2f0dc4e894a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:06.672587 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 188/378] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 10 12:23:06 2022] GET /identity/v3/role_assignments?group.id=2cfd468a091f4dbba6eaa9d630e9ec6d&scope.project.id=b287589e923744ec9ed8d9c9f4196506&role.id=0f58d2782a2840648bbbc2ab26ec02df => generated 260 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:06.723256 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156249d5-1dec-4579-828e-30265b53c527 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:06.743870 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:06.762416 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6cad80-3ff3-456f-8376-f54457ebd3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:06.763438 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6cad80-3ff3-456f-8376-f54457ebd3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:06.765406 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6cad80-3ff3-456f-8376-f54457ebd3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:06.765406 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6cad80-3ff3-456f-8376-f54457ebd3b0 None admin] PATH_INFO: `/v3/groups/e87afcd2af4b484e88e1debe318182c1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:06.770853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6cad80-3ff3-456f-8376-f54457ebd3b0 None admin] RBAC: Authorizing `identity:get_group(group_id=e87afcd2af4b484e88e1debe318182c1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:06.772886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6cad80-3ff3-456f-8376-f54457ebd3b0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:06.775265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 191/379] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 12:23:06 2022] GET /identity/v3/groups/e87afcd2af4b484e88e1debe318182c1 => generated 229 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:23:06.787209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f48e65-acdd-4368-a72b-c2f0dc4e894a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:06.861096 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6354536-fdaa-4445-9047-944cba0ab453 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:06.862291 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6354536-fdaa-4445-9047-944cba0ab453 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:06.862762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6354536-fdaa-4445-9047-944cba0ab453 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:06.865071 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 192/380] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:06.874232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df59018-53ae-4794-9b80-a885899d4140 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:06.874833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df59018-53ae-4794-9b80-a885899d4140 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:06.875353 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df59018-53ae-4794-9b80-a885899d4140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:06.897806 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-2df59018-53ae-4794-9b80-a885899d4140 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:06.899427 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2df59018-53ae-4794-9b80-a885899d4140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:06.955247 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f48e65-acdd-4368-a72b-c2f0dc4e894a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:06.987143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 193/381] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:23:06.999956 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9990149-dc87-47e0-96ed-62e8eff1603e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:07.000570 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9990149-dc87-47e0-96ed-62e8eff1603e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:07.000848 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9990149-dc87-47e0-96ed-62e8eff1603e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:07.001181 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9990149-dc87-47e0-96ed-62e8eff1603e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:07.001961 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9990149-dc87-47e0-96ed-62e8eff1603e None admin] RBAC: Adding query filter params (group.id=e87afcd2af4b484e88e1debe318182c1, role.id=bffbe227cda1418cbb58394ddff6f861, scope.project.id=3c65e78604254a1dbcd377669ee26423) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:07.002272 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9990149-dc87-47e0-96ed-62e8eff1603e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:07.003746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9990149-dc87-47e0-96ed-62e8eff1603e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:07.009685 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 189/382] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 10 12:23:06 2022] GET /identity/v3/role_assignments?group.id=e87afcd2af4b484e88e1debe318182c1&scope.project.id=3c65e78604254a1dbcd377669ee26423&role.id=bffbe227cda1418cbb58394ddff6f861 => generated 260 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:07.017830 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df59018-53ae-4794-9b80-a885899d4140 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:07.131406 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df59018-53ae-4794-9b80-a885899d4140 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:07.147461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:07.171627 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd4984-aa02-4a0f-aa79-1628baedcdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:07.181799 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd4984-aa02-4a0f-aa79-1628baedcdaf None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:07.181799 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd4984-aa02-4a0f-aa79-1628baedcdaf None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:07.181799 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd4984-aa02-4a0f-aa79-1628baedcdaf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:07.184431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd4984-aa02-4a0f-aa79-1628baedcdaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:07.189587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-7fcd4984-aa02-4a0f-aa79-1628baedcdaf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:23:07.189587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:07.189587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:07.189587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:07.189587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:07.189587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:07.189587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:07.189587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:07.189587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:07.189587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:07.189587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:07.189587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:07.189587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:07.189587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:07.189587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:07.189587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:07.190704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:07.190704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:07.190704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:07.190704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:07.190704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:07.190704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:07.190704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:07.190704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:07.190704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:07.190704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:07.190704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:07.190704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:07.190704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:07.190704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:07.190704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:07.190704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:07.191524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:07.191524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:07.191524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:07.191524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:07.191524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:07.191524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:07.191524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:07.191524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:07.191524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:07.191524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:07.191524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:23:07.191524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:07.191524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 194/383] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:23:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:07.200309 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9990149-dc87-47e0-96ed-62e8eff1603e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:07.232288 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:07.311618 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9990149-dc87-47e0-96ed-62e8eff1603e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:07.349832 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771f4a0-e98d-4a94-91a2-22eab18069c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:07.351284 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771f4a0-e98d-4a94-91a2-22eab18069c9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:07.351611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771f4a0-e98d-4a94-91a2-22eab18069c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:07.351905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771f4a0-e98d-4a94-91a2-22eab18069c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:07.352577 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771f4a0-e98d-4a94-91a2-22eab18069c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:07.352889 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771f4a0-e98d-4a94-91a2-22eab18069c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:07.354279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771f4a0-e98d-4a94-91a2-22eab18069c9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:07.363466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 190/384] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 12:23:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:07.369251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd4984-aa02-4a0f-aa79-1628baedcdaf None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:07.458161 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de461dea-4513-4ae0-8eda-1502bc3cd00c None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:07.458493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de461dea-4513-4ae0-8eda-1502bc3cd00c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:07.458715 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de461dea-4513-4ae0-8eda-1502bc3cd00c None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:07.461349 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 191/385] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:23:07.466519 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654740e5-8a1b-4ea5-b07c-ab89e90f438c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:07.466807 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654740e5-8a1b-4ea5-b07c-ab89e90f438c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:07.467087 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654740e5-8a1b-4ea5-b07c-ab89e90f438c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:07.488865 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-654740e5-8a1b-4ea5-b07c-ab89e90f438c None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:07.490230 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-654740e5-8a1b-4ea5-b07c-ab89e90f438c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:07.503190 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd4984-aa02-4a0f-aa79-1628baedcdaf None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:07.573449 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 192/386] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:07.596951 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654740e5-8a1b-4ea5-b07c-ab89e90f438c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:07.599081 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e35ecc-84ca-42a7-833c-53e9836caddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:07.600133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e35ecc-84ca-42a7-833c-53e9836caddf None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:07.600770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e35ecc-84ca-42a7-833c-53e9836caddf None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:07.601252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e35ecc-84ca-42a7-833c-53e9836caddf None admin] PATH_INFO: `/v3/groups/2cfd468a091f4dbba6eaa9d630e9ec6d` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:07.607909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e35ecc-84ca-42a7-833c-53e9836caddf None admin] RBAC: Authorizing `identity:get_group(group_id=2cfd468a091f4dbba6eaa9d630e9ec6d)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:07.609661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e35ecc-84ca-42a7-833c-53e9836caddf None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:07.611904 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 195/387] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 12:23:07 2022] GET /identity/v3/groups/2cfd468a091f4dbba6eaa9d630e9ec6d => generated 226 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:07.616909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e35ecc-84ca-42a7-833c-53e9836caddf None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:07.624690 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:07.687213 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:07.709696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654740e5-8a1b-4ea5-b07c-ab89e90f438c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:07.750000 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b0558-2ea6-45e8-b18a-8ce0f74f026e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:07.750619 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b0558-2ea6-45e8-b18a-8ce0f74f026e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:07.750905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b0558-2ea6-45e8-b18a-8ce0f74f026e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:07.755197 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b0558-2ea6-45e8-b18a-8ce0f74f026e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:07.775080 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b0558-2ea6-45e8-b18a-8ce0f74f026e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:07.780164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f65b0558-2ea6-45e8-b18a-8ce0f74f026e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:23:07.780164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:07.780164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:07.780164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:07.780164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:07.780164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:07.780164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:07.780164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:07.780164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:07.780164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:07.780164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:07.780164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:07.780164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:07.780164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:07.780164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:07.780164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:07.781067 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:07.781067 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:07.781067 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:07.781067 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:07.781067 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:07.781067 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:07.781067 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:07.781067 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:07.781067 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:07.781067 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:07.781067 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:07.781067 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:07.781067 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:07.781067 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:07.781067 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:07.781067 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:07.781865 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:07.781865 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:07.781865 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:07.781865 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:07.781865 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:07.781865 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:07.781865 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:07.781865 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:07.781865 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:07.781865 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:07.781865 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:23:07.781865 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:07.789224 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 193/388] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 12:23:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:07.792890 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b0558-2ea6-45e8-b18a-8ce0f74f026e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:07.823800 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e35ecc-84ca-42a7-833c-53e9836caddf None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:07.899674 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8e8a4-7185-4398-a703-5f4a6fe3fec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:07.900713 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8e8a4-7185-4398-a703-5f4a6fe3fec2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:07.901207 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8e8a4-7185-4398-a703-5f4a6fe3fec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:07.901677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8e8a4-7185-4398-a703-5f4a6fe3fec2 None admin] PATH_INFO: `/v3/projects/9c9a0023662b41afa1d7bec05fd12eda` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:07.903080 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b8e8a4-7185-4398-a703-5f4a6fe3fec2 None admin] RBAC: Authorizing `identity:get_project(project_id=9c9a0023662b41afa1d7bec05fd12eda)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:07.904767 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b8e8a4-7185-4398-a703-5f4a6fe3fec2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:07.906647 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 196/389] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:23:07 2022] GET /identity/v3/projects/9c9a0023662b41afa1d7bec05fd12eda => generated 298 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:23:07.912232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8e8a4-7185-4398-a703-5f4a6fe3fec2 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:07.915587 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:07.935963 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b0558-2ea6-45e8-b18a-8ce0f74f026e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:07.977037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf9f95-df3a-42d2-a1f9-b4e3bf5cd50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:07.978009 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf9f95-df3a-42d2-a1f9-b4e3bf5cd50d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:07.978121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf9f95-df3a-42d2-a1f9-b4e3bf5cd50d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:07.978375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf9f95-df3a-42d2-a1f9-b4e3bf5cd50d None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:07.979050 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf9f95-df3a-42d2-a1f9-b4e3bf5cd50d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:07.979407 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf9f95-df3a-42d2-a1f9-b4e3bf5cd50d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:07.980856 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf9f95-df3a-42d2-a1f9-b4e3bf5cd50d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:07.990376 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 194/390] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:07.997561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf9f95-df3a-42d2-a1f9-b4e3bf5cd50d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:08.117633 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf9f95-df3a-42d2-a1f9-b4e3bf5cd50d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:08.128943 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8e8a4-7185-4398-a703-5f4a6fe3fec2 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:08.144688 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:08.159318 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37e5ee-7059-4578-981d-4fa3313f4336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:08.165405 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37e5ee-7059-4578-981d-4fa3313f4336 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:08.165405 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37e5ee-7059-4578-981d-4fa3313f4336 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:08.165405 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37e5ee-7059-4578-981d-4fa3313f4336 None admin] PATH_INFO: `/v3/groups/2cfd468a091f4dbba6eaa9d630e9ec6d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:08.165405 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d37e5ee-7059-4578-981d-4fa3313f4336 None admin] RBAC: Authorizing `identity:get_group(group_id=2cfd468a091f4dbba6eaa9d630e9ec6d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:08.165405 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d37e5ee-7059-4578-981d-4fa3313f4336 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:08.175225 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 195/391] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 12:23:07 2022] GET /identity/v3/groups/2cfd468a091f4dbba6eaa9d630e9ec6d => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:23:08.175225 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37e5ee-7059-4578-981d-4fa3313f4336 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:08.186521 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c0677-fd5c-49db-a9eb-4516279a3600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:08.187571 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:08.188066 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c0677-fd5c-49db-a9eb-4516279a3600 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:08.188549 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c0677-fd5c-49db-a9eb-4516279a3600 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:08.189008 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c0677-fd5c-49db-a9eb-4516279a3600 None admin] PATH_INFO: `/v3/projects/9c9a0023662b41afa1d7bec05fd12eda/groups/2cfd468a091f4dbba6eaa9d630e9ec6d/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:08.196562 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212c0677-fd5c-49db-a9eb-4516279a3600 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c9a0023662b41afa1d7bec05fd12eda, group_id=2cfd468a091f4dbba6eaa9d630e9ec6d, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:08.198267 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212c0677-fd5c-49db-a9eb-4516279a3600 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:08.285735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37e5ee-7059-4578-981d-4fa3313f4336 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:08.304448 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 197/392] 10.222.0.49 () {58 vars in 1442 bytes} [Wed Aug 10 12:23:07 2022] PUT /identity/v3/projects/9c9a0023662b41afa1d7bec05fd12eda/groups/2cfd468a091f4dbba6eaa9d630e9ec6d/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:23:08.324811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d76122-7c50-42bd-984a-5ebf60990921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:08.325461 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d76122-7c50-42bd-984a-5ebf60990921 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:08.325825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d76122-7c50-42bd-984a-5ebf60990921 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:08.326106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d76122-7c50-42bd-984a-5ebf60990921 None admin] PATH_INFO: `/v3/projects/9c9a0023662b41afa1d7bec05fd12eda` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:08.327283 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d76122-7c50-42bd-984a-5ebf60990921 None admin] RBAC: Authorizing `identity:get_project(project_id=9c9a0023662b41afa1d7bec05fd12eda)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:08.328848 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d76122-7c50-42bd-984a-5ebf60990921 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:08.330991 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 196/393] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:23:08 2022] GET /identity/v3/projects/9c9a0023662b41afa1d7bec05fd12eda => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:08.337582 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c0677-fd5c-49db-a9eb-4516279a3600 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:08.454067 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c0677-fd5c-49db-a9eb-4516279a3600 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:08.498083 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52094bcc-94d7-4bf9-b6fa-5eb49db64560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:08.498768 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52094bcc-94d7-4bf9-b6fa-5eb49db64560 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:08.499048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52094bcc-94d7-4bf9-b6fa-5eb49db64560 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:08.499360 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52094bcc-94d7-4bf9-b6fa-5eb49db64560 None admin] PATH_INFO: `/v3/projects/9c9a0023662b41afa1d7bec05fd12eda/groups/2cfd468a091f4dbba6eaa9d630e9ec6d/roles/0f58d2782a2840648bbbc2ab26ec02df` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:08.501651 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52094bcc-94d7-4bf9-b6fa-5eb49db64560 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c9a0023662b41afa1d7bec05fd12eda, group_id=2cfd468a091f4dbba6eaa9d630e9ec6d, role_id=0f58d2782a2840648bbbc2ab26ec02df)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:08.503307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52094bcc-94d7-4bf9-b6fa-5eb49db64560 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:08.585865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 198/394] 10.222.0.49 () {58 vars in 1442 bytes} [Wed Aug 10 12:23:08 2022] PUT /identity/v3/projects/9c9a0023662b41afa1d7bec05fd12eda/groups/2cfd468a091f4dbba6eaa9d630e9ec6d/roles/0f58d2782a2840648bbbc2ab26ec02df => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:23:08.632379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba736cc-14d6-441c-9323-8aab5b66aa73 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:08.632942 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba736cc-14d6-441c-9323-8aab5b66aa73 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:08.633432 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba736cc-14d6-441c-9323-8aab5b66aa73 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:08.635017 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 197/395] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:08.640514 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d548834-dd56-49d7-9864-ddbfbd32534a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:08.640797 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d548834-dd56-49d7-9864-ddbfbd32534a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:08.641069 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d548834-dd56-49d7-9864-ddbfbd32534a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:08.665510 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-4d548834-dd56-49d7-9864-ddbfbd32534a None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:08.666718 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4d548834-dd56-49d7-9864-ddbfbd32534a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:08.695754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:08.755770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 199/396] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:08.793950 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba736cc-14d6-441c-9323-8aab5b66aa73 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:08.895092 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:08.922969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba736cc-14d6-441c-9323-8aab5b66aa73 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:08.969053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0b7df-292e-452a-813a-279ab67312f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:08.969708 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0b7df-292e-452a-813a-279ab67312f1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:08.969992 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0b7df-292e-452a-813a-279ab67312f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:08.970267 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0b7df-292e-452a-813a-279ab67312f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:08.987461 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c0b7df-292e-452a-813a-279ab67312f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:08.992843 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-61c0b7df-292e-452a-813a-279ab67312f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:23:08.992843 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:08.992843 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:08.992843 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:08.992843 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:08.992843 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:08.992843 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:08.992843 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:08.992843 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:08.992843 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:08.992843 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:08.992843 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:08.992843 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:08.992843 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:08.992843 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:08.992843 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:08.996433 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:08.996433 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:08.996433 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:08.996433 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:08.996433 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:08.996433 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:08.996433 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:08.996433 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:08.996433 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:08.996433 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:08.996433 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:08.996433 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:08.996433 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:08.996433 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:08.996433 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:08.996433 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:08.997281 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:08.997281 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:08.997281 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:08.997281 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:08.997281 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:08.997281 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:08.997281 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:08.997281 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:08.997281 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:08.997281 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:08.997281 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:23:08.997281 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:08.997281 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 198/397] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:23:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:23:09.003286 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d548834-dd56-49d7-9864-ddbfbd32534a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:09.139346 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064e6e3-7844-46e1-b87b-d77fb3987496 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:09.139842 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064e6e3-7844-46e1-b87b-d77fb3987496 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:09.139842 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064e6e3-7844-46e1-b87b-d77fb3987496 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:09.142321 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 199/398] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:23:09.146185 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613e065-217a-48e6-a6b5-186b4f5a6c15 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:09.146472 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613e065-217a-48e6-a6b5-186b4f5a6c15 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:09.146825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613e065-217a-48e6-a6b5-186b4f5a6c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:09.150371 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d548834-dd56-49d7-9864-ddbfbd32534a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:09.182876 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4613e065-217a-48e6-a6b5-186b4f5a6c15 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:09.185396 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4613e065-217a-48e6-a6b5-186b4f5a6c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:09.210036 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ce5af-7283-4946-b120-8880dfe70fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:09.211370 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ce5af-7283-4946-b120-8880dfe70fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:09.211859 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ce5af-7283-4946-b120-8880dfe70fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:09.212321 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ce5af-7283-4946-b120-8880dfe70fe0 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:09.213214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ce5af-7283-4946-b120-8880dfe70fe0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:09.213716 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ce5af-7283-4946-b120-8880dfe70fe0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:09.215493 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ce5af-7283-4946-b120-8880dfe70fe0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:09.234355 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 200/399] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 12:23:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:09.241941 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ce5af-7283-4946-b120-8880dfe70fe0 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:09.290721 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:09.296166 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 200/400] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:23:09.337664 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613e065-217a-48e6-a6b5-186b4f5a6c15 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:09.435533 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ce5af-7283-4946-b120-8880dfe70fe0 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:09.464367 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:09.497015 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5655d2-5570-4b87-a79f-d228dc99aed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:09.497660 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5655d2-5570-4b87-a79f-d228dc99aed0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:09.497936 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5655d2-5570-4b87-a79f-d228dc99aed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:09.498205 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5655d2-5570-4b87-a79f-d228dc99aed0 None admin] PATH_INFO: `/v3/groups/2cfd468a091f4dbba6eaa9d630e9ec6d` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:09.499462 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5655d2-5570-4b87-a79f-d228dc99aed0 None admin] RBAC: Authorizing `identity:get_group(group_id=2cfd468a091f4dbba6eaa9d630e9ec6d)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:09.500877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5655d2-5570-4b87-a79f-d228dc99aed0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:09.502651 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 201/401] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 12:23:09 2022] GET /identity/v3/groups/2cfd468a091f4dbba6eaa9d630e9ec6d => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:23:09.511619 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5655d2-5570-4b87-a79f-d228dc99aed0 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:09.532730 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613e065-217a-48e6-a6b5-186b4f5a6c15 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:09.592584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eefa18-574c-4f8f-b09a-c4cb548f61ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:09.593230 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eefa18-574c-4f8f-b09a-c4cb548f61ab None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:09.593512 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eefa18-574c-4f8f-b09a-c4cb548f61ab None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:09.593785 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eefa18-574c-4f8f-b09a-c4cb548f61ab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:09.605015 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eefa18-574c-4f8f-b09a-c4cb548f61ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:09.609968 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d3eefa18-574c-4f8f-b09a-c4cb548f61ab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:23:09.609968 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:09.609968 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:09.609968 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:09.609968 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:09.609968 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:09.609968 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:09.609968 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:09.609968 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:09.609968 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:09.609968 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:09.609968 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:09.609968 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:09.609968 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:09.609968 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:09.609968 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:09.610858 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:09.610858 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:09.610858 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:09.610858 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:09.610858 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:09.610858 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:09.610858 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:09.610858 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:09.610858 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:09.610858 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:09.610858 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:09.610858 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:09.610858 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:09.610858 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:09.610858 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:09.610858 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:09.612505 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:09.612505 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:09.612505 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:09.612505 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:09.612505 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:09.612505 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:09.612505 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:09.612505 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:09.612505 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:09.612505 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:09.612505 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:23:09.612505 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:09.612505 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 201/402] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 12:23:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:23:09.618283 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eefa18-574c-4f8f-b09a-c4cb548f61ab None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:09.681375 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:09.708559 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5655d2-5570-4b87-a79f-d228dc99aed0 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:09.729506 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eefa18-574c-4f8f-b09a-c4cb548f61ab None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:09.768667 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:09.768865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8794c6c-2c45-435f-9731-da56e3bf6aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:09.768865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8794c6c-2c45-435f-9731-da56e3bf6aaa None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:09.768865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8794c6c-2c45-435f-9731-da56e3bf6aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:09.768865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8794c6c-2c45-435f-9731-da56e3bf6aaa None admin] PATH_INFO: `/v3/projects/b287589e923744ec9ed8d9c9f4196506` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:09.769212 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8794c6c-2c45-435f-9731-da56e3bf6aaa None admin] RBAC: Authorizing `identity:get_project(project_id=b287589e923744ec9ed8d9c9f4196506)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:09.769212 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8794c6c-2c45-435f-9731-da56e3bf6aaa None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:09.771254 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 202/403] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:23:09 2022] GET /identity/v3/projects/b287589e923744ec9ed8d9c9f4196506 => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:09.778491 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8794c6c-2c45-435f-9731-da56e3bf6aaa None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:09.780967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3728ac-744e-49ff-83b8-55753cc281d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:09.782185 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3728ac-744e-49ff-83b8-55753cc281d2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:09.784341 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3728ac-744e-49ff-83b8-55753cc281d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:09.784792 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3728ac-744e-49ff-83b8-55753cc281d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:09.785619 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3728ac-744e-49ff-83b8-55753cc281d2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:09.786161 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3728ac-744e-49ff-83b8-55753cc281d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:09.789171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3728ac-744e-49ff-83b8-55753cc281d2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:09.808171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 202/404] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:09.816211 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3728ac-744e-49ff-83b8-55753cc281d2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:09.951805 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8794c6c-2c45-435f-9731-da56e3bf6aaa None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:09.970764 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3728ac-744e-49ff-83b8-55753cc281d2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:09.991617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a9ae4-e026-4d27-8056-96e07065eb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:09.992602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a9ae4-e026-4d27-8056-96e07065eb5d None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:09.992885 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a9ae4-e026-4d27-8056-96e07065eb5d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:09.993161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a9ae4-e026-4d27-8056-96e07065eb5d None admin] PATH_INFO: `/v3/projects/b287589e923744ec9ed8d9c9f4196506/groups/2cfd468a091f4dbba6eaa9d630e9ec6d/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:09.996266 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a9ae4-e026-4d27-8056-96e07065eb5d None admin] RBAC: Authorizing `identity:create_grant(project_id=b287589e923744ec9ed8d9c9f4196506, group_id=2cfd468a091f4dbba6eaa9d630e9ec6d, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:09.997714 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a9ae4-e026-4d27-8056-96e07065eb5d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:10.019341 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953d4e9-5e8e-4de1-bb27-d7606e2d7e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:10.020248 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953d4e9-5e8e-4de1-bb27-d7606e2d7e15 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:10.020699 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953d4e9-5e8e-4de1-bb27-d7606e2d7e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:10.021156 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953d4e9-5e8e-4de1-bb27-d7606e2d7e15 None admin] PATH_INFO: `/v3/groups/2cfd468a091f4dbba6eaa9d630e9ec6d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:10.022465 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953d4e9-5e8e-4de1-bb27-d7606e2d7e15 None admin] RBAC: Authorizing `identity:get_group(group_id=2cfd468a091f4dbba6eaa9d630e9ec6d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:10.024105 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953d4e9-5e8e-4de1-bb27-d7606e2d7e15 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:10.030522 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 203/405] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 12:23:09 2022] GET /identity/v3/groups/2cfd468a091f4dbba6eaa9d630e9ec6d => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:10.030522 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f26d8-f2d1-4cb0-a9eb-23c9f28a0c75 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:10.030522 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f26d8-f2d1-4cb0-a9eb-23c9f28a0c75 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:10.030522 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f26d8-f2d1-4cb0-a9eb-23c9f28a0c75 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:10.030522 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 204/406] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:10.036029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f26d8-f2d1-4cb0-a9eb-23c9f28a0c75 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:10.106593 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:10.107592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 203/407] 10.222.0.49 () {58 vars in 1442 bytes} [Wed Aug 10 12:23:09 2022] PUT /identity/v3/projects/b287589e923744ec9ed8d9c9f4196506/groups/2cfd468a091f4dbba6eaa9d630e9ec6d/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:23:10.108778 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6a666-7a0a-4d09-837a-3e4d92bc0be5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:10.109065 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6a666-7a0a-4d09-837a-3e4d92bc0be5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:10.109344 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6a666-7a0a-4d09-837a-3e4d92bc0be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:10.156441 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-1de6a666-7a0a-4d09-837a-3e4d92bc0be5 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:10.157778 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1de6a666-7a0a-4d09-837a-3e4d92bc0be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:10.168365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f26d8-f2d1-4cb0-a9eb-23c9f28a0c75 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:10.172214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:10.222432 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0237590-8bbd-4e7b-8fe5-bd5edc313b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:10.223484 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0237590-8bbd-4e7b-8fe5-bd5edc313b93 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:10.223929 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0237590-8bbd-4e7b-8fe5-bd5edc313b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:10.224538 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0237590-8bbd-4e7b-8fe5-bd5edc313b93 None admin] PATH_INFO: `/v3/projects/b287589e923744ec9ed8d9c9f4196506` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:10.227621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0237590-8bbd-4e7b-8fe5-bd5edc313b93 None admin] RBAC: Authorizing `identity:get_project(project_id=b287589e923744ec9ed8d9c9f4196506)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:10.230745 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0237590-8bbd-4e7b-8fe5-bd5edc313b93 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:10.235930 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 205/408] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:23:10 2022] GET /identity/v3/projects/b287589e923744ec9ed8d9c9f4196506 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:10.241918 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0237590-8bbd-4e7b-8fe5-bd5edc313b93 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:10.244224 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 204/409] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:10.284808 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6a666-7a0a-4d09-837a-3e4d92bc0be5 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:10.419518 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6a666-7a0a-4d09-837a-3e4d92bc0be5 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:10.423069 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0237590-8bbd-4e7b-8fe5-bd5edc313b93 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:10.471568 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:10.486908 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7e9e8-84d8-47ae-8586-25539be5c745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:10.486908 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7e9e8-84d8-47ae-8586-25539be5c745 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:10.486908 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7e9e8-84d8-47ae-8586-25539be5c745 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:10.486908 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7e9e8-84d8-47ae-8586-25539be5c745 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:10.487303 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98615392-b0c5-45c1-8dce-3456acd290b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:10.487303 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98615392-b0c5-45c1-8dce-3456acd290b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:10.487303 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98615392-b0c5-45c1-8dce-3456acd290b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:10.487303 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98615392-b0c5-45c1-8dce-3456acd290b7 None admin] PATH_INFO: `/v3/projects/b287589e923744ec9ed8d9c9f4196506/groups/2cfd468a091f4dbba6eaa9d630e9ec6d/roles/0f58d2782a2840648bbbc2ab26ec02df` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:10.490912 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98615392-b0c5-45c1-8dce-3456acd290b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=b287589e923744ec9ed8d9c9f4196506, group_id=2cfd468a091f4dbba6eaa9d630e9ec6d, role_id=0f58d2782a2840648bbbc2ab26ec02df)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:10.492537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98615392-b0c5-45c1-8dce-3456acd290b7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:10.502260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd7e9e8-84d8-47ae-8586-25539be5c745 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:10.507828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-0fd7e9e8-84d8-47ae-8586-25539be5c745 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:23:10.507828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:10.507828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:10.507828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:10.507828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:10.507828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:10.507828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:10.507828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:10.507828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:10.507828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:10.507828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:10.507828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:10.507828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:10.507828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:10.507828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:10.507828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:10.508722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:10.508722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:10.508722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:10.508722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:10.508722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:10.508722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:10.508722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:10.508722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:10.508722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:10.508722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:10.508722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:10.508722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:10.508722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:10.508722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:10.508722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:10.508722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:10.509526 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:10.509526 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:10.509526 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:10.509526 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:10.509526 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:10.509526 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:10.509526 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:10.509526 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:10.509526 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:10.509526 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:10.509526 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:23:10.509526 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:10.509526 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 205/410] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:23:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:10.515883 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7e9e8-84d8-47ae-8586-25539be5c745 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:10.543757 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:10.580328 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 206/411] 10.222.0.49 () {58 vars in 1442 bytes} [Wed Aug 10 12:23:10 2022] PUT /identity/v3/projects/b287589e923744ec9ed8d9c9f4196506/groups/2cfd468a091f4dbba6eaa9d630e9ec6d/roles/0f58d2782a2840648bbbc2ab26ec02df => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:23:10.634051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7e9e8-84d8-47ae-8586-25539be5c745 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:10.684104 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2cbcea-0877-4eaf-be68-eaaab7d97b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:10.685173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2cbcea-0877-4eaf-be68-eaaab7d97b8e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:10.685660 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2cbcea-0877-4eaf-be68-eaaab7d97b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:10.686142 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2cbcea-0877-4eaf-be68-eaaab7d97b8e None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:10.687196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2cbcea-0877-4eaf-be68-eaaab7d97b8e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:10.687756 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2cbcea-0877-4eaf-be68-eaaab7d97b8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:10.689386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2cbcea-0877-4eaf-be68-eaaab7d97b8e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:10.716880 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 206/412] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:23:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:23:10.732584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98615392-b0c5-45c1-8dce-3456acd290b7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:10.959595 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2bb73-2f09-4b9b-a86e-399e75a1326c None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:10.959965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2bb73-2f09-4b9b-a86e-399e75a1326c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:10.960182 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2bb73-2f09-4b9b-a86e-399e75a1326c None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:10.961643 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 207/413] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:10.967839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5231ef7-3ebc-47a5-b7a7-ae1a0d42331e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:10.968116 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5231ef7-3ebc-47a5-b7a7-ae1a0d42331e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:10.968477 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5231ef7-3ebc-47a5-b7a7-ae1a0d42331e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:10.992990 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98615392-b0c5-45c1-8dce-3456acd290b7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:11.033513 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a5231ef7-3ebc-47a5-b7a7-ae1a0d42331e None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:11.038118 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a5231ef7-3ebc-47a5-b7a7-ae1a0d42331e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:11.126183 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d5e8e-29f6-4e97-93d4-7418daa55e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:11.127285 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d5e8e-29f6-4e97-93d4-7418daa55e49 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:11.127787 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d5e8e-29f6-4e97-93d4-7418daa55e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:11.128291 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d5e8e-29f6-4e97-93d4-7418daa55e49 None admin] PATH_INFO: `/v3/groups/e87afcd2af4b484e88e1debe318182c1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:11.129682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428d5e8e-29f6-4e97-93d4-7418daa55e49 None admin] RBAC: Authorizing `identity:get_group(group_id=e87afcd2af4b484e88e1debe318182c1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:11.132113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428d5e8e-29f6-4e97-93d4-7418daa55e49 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:11.134234 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 207/414] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 12:23:10 2022] GET /identity/v3/groups/e87afcd2af4b484e88e1debe318182c1 => generated 229 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:11.140356 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d5e8e-29f6-4e97-93d4-7418daa55e49 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:11.234248 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:11.244115 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 208/415] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:11.266828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5231ef7-3ebc-47a5-b7a7-ae1a0d42331e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:11.323608 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:11.402278 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d5e8e-29f6-4e97-93d4-7418daa55e49 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:11.451509 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec3bcc-74ec-4e24-95ee-e1c9b35aac29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:11.452154 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec3bcc-74ec-4e24-95ee-e1c9b35aac29 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:11.452536 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec3bcc-74ec-4e24-95ee-e1c9b35aac29 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:11.452818 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec3bcc-74ec-4e24-95ee-e1c9b35aac29 None admin] PATH_INFO: `/v3/projects/3c65e78604254a1dbcd377669ee26423` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:11.454057 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec3bcc-74ec-4e24-95ee-e1c9b35aac29 None admin] RBAC: Authorizing `identity:get_project(project_id=3c65e78604254a1dbcd377669ee26423)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:11.455644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec3bcc-74ec-4e24-95ee-e1c9b35aac29 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:11.457658 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 208/416] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:23:11 2022] GET /identity/v3/projects/3c65e78604254a1dbcd377669ee26423 => generated 344 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:11.459058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cffdbf-ef6b-40ac-b93e-ae4a5c72ff04 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:11.459922 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cffdbf-ef6b-40ac-b93e-ae4a5c72ff04 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:11.460201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cffdbf-ef6b-40ac-b93e-ae4a5c72ff04 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:11.461418 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 209/417] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:11.462758 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5231ef7-3ebc-47a5-b7a7-ae1a0d42331e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:11.466115 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cffdbf-ef6b-40ac-b93e-ae4a5c72ff04 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:11.504657 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7fd91-ac36-45fa-87f4-aa7d6e71a21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:11.506724 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7fd91-ac36-45fa-87f4-aa7d6e71a21a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:11.507255 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7fd91-ac36-45fa-87f4-aa7d6e71a21a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:11.507596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7fd91-ac36-45fa-87f4-aa7d6e71a21a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:11.520183 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b7fd91-ac36-45fa-87f4-aa7d6e71a21a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:11.527280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-07b7fd91-ac36-45fa-87f4-aa7d6e71a21a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:23:11.527280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:11.527280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:11.527280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:11.527280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:11.527280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:11.527280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:11.527280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:11.527280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:11.527280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:11.527280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:11.527280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:11.527280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:11.527280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:11.527280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:11.527280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:11.528173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:11.528173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:11.528173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:11.528173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:11.528173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:11.528173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:11.528173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:11.528173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:11.528173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:11.528173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:11.528173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:11.528173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:11.528173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:11.528173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:11.528173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:11.528173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:11.529502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:11.529502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:11.529502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:11.529502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:11.529502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:11.529502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:11.529502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:11.529502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:11.529502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:11.529502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:11.529502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:23:11.529502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:11.529502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 209/418] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:23:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:23:11.531057 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257bbfe-0de9-4e86-bc2f-ee4078ff939f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:11.531057 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257bbfe-0de9-4e86-bc2f-ee4078ff939f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:11.531057 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257bbfe-0de9-4e86-bc2f-ee4078ff939f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:11.552508 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3257bbfe-0de9-4e86-bc2f-ee4078ff939f None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:11.554985 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3257bbfe-0de9-4e86-bc2f-ee4078ff939f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:11.595034 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cffdbf-ef6b-40ac-b93e-ae4a5c72ff04 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:11.618346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:11.628353 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:11.633648 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 210/419] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:11.634669 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257bbfe-0de9-4e86-bc2f-ee4078ff939f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:11.652179 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec432d6c-f66d-4ac1-b1a3-d6a76e45b814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:11.654593 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec432d6c-f66d-4ac1-b1a3-d6a76e45b814 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:11.655086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec432d6c-f66d-4ac1-b1a3-d6a76e45b814 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:11.655921 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec432d6c-f66d-4ac1-b1a3-d6a76e45b814 None admin] PATH_INFO: `/v3/projects/3c65e78604254a1dbcd377669ee26423/groups/e87afcd2af4b484e88e1debe318182c1/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:11.659559 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec432d6c-f66d-4ac1-b1a3-d6a76e45b814 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c65e78604254a1dbcd377669ee26423, group_id=e87afcd2af4b484e88e1debe318182c1, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:11.664025 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec432d6c-f66d-4ac1-b1a3-d6a76e45b814 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:11.744475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 210/420] 10.222.0.49 () {58 vars in 1442 bytes} [Wed Aug 10 12:23:11 2022] PUT /identity/v3/projects/3c65e78604254a1dbcd377669ee26423/groups/e87afcd2af4b484e88e1debe318182c1/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:23:11.746022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec432d6c-f66d-4ac1-b1a3-d6a76e45b814 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:11.762129 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257bbfe-0de9-4e86-bc2f-ee4078ff939f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:11.812837 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead403dc-ab84-4524-adf7-14e25dc8c960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:11.812837 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead403dc-ab84-4524-adf7-14e25dc8c960 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:11.812837 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead403dc-ab84-4524-adf7-14e25dc8c960 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:11.813210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead403dc-ab84-4524-adf7-14e25dc8c960 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:11.813820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead403dc-ab84-4524-adf7-14e25dc8c960 None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:11.814127 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead403dc-ab84-4524-adf7-14e25dc8c960 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:11.815633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead403dc-ab84-4524-adf7-14e25dc8c960 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:11.830664 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 211/421] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 12:23:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:11.840294 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead403dc-ab84-4524-adf7-14e25dc8c960 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:11.930901 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:11.939304 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec432d6c-f66d-4ac1-b1a3-d6a76e45b814 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:11.992695 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e796ec7-bbf7-48a5-aa11-38f7bad842cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:11.993342 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e796ec7-bbf7-48a5-aa11-38f7bad842cf None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:11.993621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e796ec7-bbf7-48a5-aa11-38f7bad842cf None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:11.993895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e796ec7-bbf7-48a5-aa11-38f7bad842cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:11.997205 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead403dc-ab84-4524-adf7-14e25dc8c960 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:12.010575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e796ec7-bbf7-48a5-aa11-38f7bad842cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:12.023201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-3e796ec7-bbf7-48a5-aa11-38f7bad842cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:23:12.023201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:12.023201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:12.023201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:12.023201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:12.023201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:12.023201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:12.023201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:12.023201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:12.023201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:12.023201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:12.023201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:12.023201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:12.023201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:12.023201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:12.023201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:12.024159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:12.024159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:12.024159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:12.024159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:12.024159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:12.024159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:12.024159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:12.024159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:12.024159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:12.024159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:12.024159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:12.024159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:12.024159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:12.024159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:12.024159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:12.024159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:12.024948 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:12.024948 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:12.024948 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:12.024948 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:12.024948 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:12.024948 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:12.024948 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:12.024948 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:12.024948 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:12.024948 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:12.024948 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:23:12.024948 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:12.024948 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 211/422] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 12:23:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:12.030569 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e796ec7-bbf7-48a5-aa11-38f7bad842cf None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:12.038743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e9f5fa-e81f-46d2-8845-d9805ee97a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:12.039702 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9f5fa-e81f-46d2-8845-d9805ee97a6e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:12.040196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9f5fa-e81f-46d2-8845-d9805ee97a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:12.040663 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9f5fa-e81f-46d2-8845-d9805ee97a6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:12.096851 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:12.116892 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e9f5fa-e81f-46d2-8845-d9805ee97a6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:12.118687 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e9f5fa-e81f-46d2-8845-d9805ee97a6e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:12.173182 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:12.225634 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e796ec7-bbf7-48a5-aa11-38f7bad842cf None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:12.229207 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 212/423] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:23:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 12:23:12.234923 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e9f5fa-e81f-46d2-8845-d9805ee97a6e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:12.335913 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309c562-22cc-4295-8f65-fd69874f2b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:12.336574 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309c562-22cc-4295-8f65-fd69874f2b8a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:12.336822 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309c562-22cc-4295-8f65-fd69874f2b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:12.337324 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309c562-22cc-4295-8f65-fd69874f2b8a None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:12.337770 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309c562-22cc-4295-8f65-fd69874f2b8a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:12.338081 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309c562-22cc-4295-8f65-fd69874f2b8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:12.339504 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309c562-22cc-4295-8f65-fd69874f2b8a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:12.340963 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e9f5fa-e81f-46d2-8845-d9805ee97a6e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:12.366325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 212/424] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:23:12.372194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309c562-22cc-4295-8f65-fd69874f2b8a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:12.378023 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1bf0b-0a1c-4134-be8c-968d05e13927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:12.378922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1bf0b-0a1c-4134-be8c-968d05e13927 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:12.379470 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1bf0b-0a1c-4134-be8c-968d05e13927 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:12.379944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1bf0b-0a1c-4134-be8c-968d05e13927 None admin] PATH_INFO: `/v3/projects/9c9a0023662b41afa1d7bec05fd12eda` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:12.381731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1bf0b-0a1c-4134-be8c-968d05e13927 None admin] RBAC: Authorizing `identity:get_project(project_id=9c9a0023662b41afa1d7bec05fd12eda)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:12.383414 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1bf0b-0a1c-4134-be8c-968d05e13927 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:12.385566 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 213/425] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:23:12 2022] GET /identity/v3/projects/9c9a0023662b41afa1d7bec05fd12eda => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:23:12.391112 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1bf0b-0a1c-4134-be8c-968d05e13927 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:12.414936 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:12.496788 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1bf0b-0a1c-4134-be8c-968d05e13927 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:12.532683 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a6be8-b960-4277-8302-b55b2fafa830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:12.533672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a6be8-b960-4277-8302-b55b2fafa830 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:12.534167 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a6be8-b960-4277-8302-b55b2fafa830 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:12.534635 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a6be8-b960-4277-8302-b55b2fafa830 None admin] PATH_INFO: `/v3/groups/2cfd468a091f4dbba6eaa9d630e9ec6d` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:12.540384 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23a6be8-b960-4277-8302-b55b2fafa830 None admin] RBAC: Authorizing `identity:get_group(group_id=2cfd468a091f4dbba6eaa9d630e9ec6d)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:12.542814 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23a6be8-b960-4277-8302-b55b2fafa830 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:12.547675 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 214/426] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 12:23:12 2022] GET /identity/v3/groups/2cfd468a091f4dbba6eaa9d630e9ec6d => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:23:12.554280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a6be8-b960-4277-8302-b55b2fafa830 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:12.593465 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309c562-22cc-4295-8f65-fd69874f2b8a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:12.601099 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:12.665579 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:12.668128 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b47823-98f9-4fa6-8cb8-334d1c786f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:12.668756 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b47823-98f9-4fa6-8cb8-334d1c786f1b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:12.669027 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b47823-98f9-4fa6-8cb8-334d1c786f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:12.669301 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b47823-98f9-4fa6-8cb8-334d1c786f1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:12.673942 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a6be8-b960-4277-8302-b55b2fafa830 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:12.724710 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d14cf-08fc-43ed-85e7-678d296a244c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:12.726274 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d14cf-08fc-43ed-85e7-678d296a244c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:12.726734 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d14cf-08fc-43ed-85e7-678d296a244c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:12.727204 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d14cf-08fc-43ed-85e7-678d296a244c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:12.728126 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d14cf-08fc-43ed-85e7-678d296a244c None admin] RBAC: Adding query filter params (group.id=2cfd468a091f4dbba6eaa9d630e9ec6d, role.id=872ba0c4212a4684b539a68fc2c20b25, scope.project.id=9c9a0023662b41afa1d7bec05fd12eda) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:12.729122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d14cf-08fc-43ed-85e7-678d296a244c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:12.731058 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d14cf-08fc-43ed-85e7-678d296a244c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:12.740607 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 215/427] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 10 12:23:12 2022] GET /identity/v3/role_assignments?group.id=2cfd468a091f4dbba6eaa9d630e9ec6d&scope.project.id=9c9a0023662b41afa1d7bec05fd12eda&role.id=872ba0c4212a4684b539a68fc2c20b25 => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:12.747419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086e992-300a-4af0-afbd-2693f14f9fe1 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:12.747419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086e992-300a-4af0-afbd-2693f14f9fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:12.747419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086e992-300a-4af0-afbd-2693f14f9fe1 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:12.753135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 216/428] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:12.755465 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e82ec1-b818-4e9c-9aaf-88e142375d38 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:12.755754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e82ec1-b818-4e9c-9aaf-88e142375d38 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:12.756559 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e82ec1-b818-4e9c-9aaf-88e142375d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:12.760115 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b47823-98f9-4fa6-8cb8-334d1c786f1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:12.761607 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b47823-98f9-4fa6-8cb8-334d1c786f1b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:12.794501 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a0e82ec1-b818-4e9c-9aaf-88e142375d38 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:12.797102 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a0e82ec1-b818-4e9c-9aaf-88e142375d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:12.944680 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 217/429] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:12.962266 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836972b-aa92-42cb-9175-65d9ae2968f1 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:12.962551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836972b-aa92-42cb-9175-65d9ae2968f1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:12.966248 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836972b-aa92-42cb-9175-65d9ae2968f1 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:12.966248 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 218/430] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:23:12.966893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 213/431] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:23:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:23:12.969624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00058862-a559-4317-a04d-7faa41427083 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:12.969912 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00058862-a559-4317-a04d-7faa41427083 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:12.970190 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00058862-a559-4317-a04d-7faa41427083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:12.972051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b47823-98f9-4fa6-8cb8-334d1c786f1b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:12.985053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:12.992026 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-00058862-a559-4317-a04d-7faa41427083 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:12.995562 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-00058862-a559-4317-a04d-7faa41427083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:13.077091 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 219/432] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:13.077929 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00058862-a559-4317-a04d-7faa41427083 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:13.093201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b47823-98f9-4fa6-8cb8-334d1c786f1b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:13.094795 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:13.133656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf24fa-a739-438f-a569-57272787304e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:13.134546 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf24fa-a739-438f-a569-57272787304e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:13.134826 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf24fa-a739-438f-a569-57272787304e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:13.135098 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf24fa-a739-438f-a569-57272787304e None admin] PATH_INFO: `/v3/projects/9c9a0023662b41afa1d7bec05fd12eda` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:13.136269 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf24fa-a739-438f-a569-57272787304e None admin] RBAC: Authorizing `identity:get_project(project_id=9c9a0023662b41afa1d7bec05fd12eda)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:13.137717 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf24fa-a739-438f-a569-57272787304e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:13.139680 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 214/433] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:23:12 2022] GET /identity/v3/projects/9c9a0023662b41afa1d7bec05fd12eda => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:13.140524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf24fa-a739-438f-a569-57272787304e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:13.220146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00058862-a559-4317-a04d-7faa41427083 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:13.227751 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:13.270883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf24fa-a739-438f-a569-57272787304e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:13.275446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c690c84f-e1a8-4377-8a2a-4215afe4a56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:13.276379 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690c84f-e1a8-4377-8a2a-4215afe4a56a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:13.276851 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690c84f-e1a8-4377-8a2a-4215afe4a56a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:13.277307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690c84f-e1a8-4377-8a2a-4215afe4a56a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:13.290559 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c690c84f-e1a8-4377-8a2a-4215afe4a56a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:13.297459 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-c690c84f-e1a8-4377-8a2a-4215afe4a56a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:23:13.297459 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:13.297459 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:13.297459 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:13.297459 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:13.297459 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:13.297459 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:13.297459 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:13.297459 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:13.297459 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:13.297459 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:13.297459 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:13.297459 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:13.297459 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:13.297459 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:13.297459 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:13.298366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:13.298366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:13.298366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:13.298366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:13.298366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:13.298366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:13.298366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:13.298366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:13.298366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:13.298366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:13.298366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:13.298366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:13.298366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:13.298366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:13.298366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:13.298366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:13.299169 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:13.299169 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:13.299169 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:13.299169 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:13.299169 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:13.299169 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:13.299169 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:13.299169 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:13.299169 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:13.299169 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:13.299169 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:23:13.299169 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:13.300875 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 220/434] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:23:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:13.306416 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c690c84f-e1a8-4377-8a2a-4215afe4a56a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:13.313952 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c4d3b-f174-460b-8fa9-992249fae015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:13.314591 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c4d3b-f174-460b-8fa9-992249fae015 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:13.314872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c4d3b-f174-460b-8fa9-992249fae015 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:13.315145 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c4d3b-f174-460b-8fa9-992249fae015 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:13.327917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c4d3b-f174-460b-8fa9-992249fae015 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:13.333991 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-2f1c4d3b-f174-460b-8fa9-992249fae015 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:23:13.333991 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:13.333991 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:13.333991 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:13.333991 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:13.333991 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:13.333991 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:13.333991 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:13.333991 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:13.333991 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:13.333991 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:13.333991 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:13.333991 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:13.333991 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:13.333991 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:13.333991 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:13.335106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:13.335106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:13.335106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:13.335106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:13.335106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:13.335106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:13.335106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:13.335106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:13.335106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:13.335106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:13.335106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:13.335106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:13.335106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:13.335106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:13.335106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:13.335106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:13.336008 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:13.336008 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:13.336008 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:13.336008 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:13.336008 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:13.336008 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:13.336008 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:13.336008 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:13.336008 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:13.336008 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:13.336008 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:23:13.336008 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:13.336008 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 215/435] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 12:23:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:13.336008 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c4d3b-f174-460b-8fa9-992249fae015 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:13.389157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:13.419913 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c690c84f-e1a8-4377-8a2a-4215afe4a56a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:13.446580 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c4d3b-f174-460b-8fa9-992249fae015 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:13.457036 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7309e1c2-0cdc-47f1-90bf-f35a3f57348b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:13.457673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7309e1c2-0cdc-47f1-90bf-f35a3f57348b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:13.457946 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7309e1c2-0cdc-47f1-90bf-f35a3f57348b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:13.458223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7309e1c2-0cdc-47f1-90bf-f35a3f57348b None admin] PATH_INFO: `/v3/groups/2cfd468a091f4dbba6eaa9d630e9ec6d` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:13.459419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7309e1c2-0cdc-47f1-90bf-f35a3f57348b None admin] RBAC: Authorizing `identity:get_group(group_id=2cfd468a091f4dbba6eaa9d630e9ec6d)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:13.460977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7309e1c2-0cdc-47f1-90bf-f35a3f57348b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:13.462855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 221/436] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 12:23:13 2022] GET /identity/v3/groups/2cfd468a091f4dbba6eaa9d630e9ec6d => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:13.463650 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7309e1c2-0cdc-47f1-90bf-f35a3f57348b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:13.483066 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:13.488694 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bbc602-d336-448e-908f-16c2437792ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:13.489342 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbc602-d336-448e-908f-16c2437792ff None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:13.489583 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbc602-d336-448e-908f-16c2437792ff None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:13.489866 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbc602-d336-448e-908f-16c2437792ff None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:13.491311 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bbc602-d336-448e-908f-16c2437792ff None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:13.497199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bbc602-d336-448e-908f-16c2437792ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:13.497199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bbc602-d336-448e-908f-16c2437792ff None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:13.502181 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 216/437] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 12:23:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:13.507792 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bbc602-d336-448e-908f-16c2437792ff None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:13.572499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7309e1c2-0cdc-47f1-90bf-f35a3f57348b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:13.608109 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f05f88-4e6d-4938-ba03-638e902454d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:13.608637 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f05f88-4e6d-4938-ba03-638e902454d0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:13.608919 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f05f88-4e6d-4938-ba03-638e902454d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:13.609194 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f05f88-4e6d-4938-ba03-638e902454d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:13.609850 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f05f88-4e6d-4938-ba03-638e902454d0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:13.610152 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f05f88-4e6d-4938-ba03-638e902454d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:13.611589 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f05f88-4e6d-4938-ba03-638e902454d0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:13.616507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bbc602-d336-448e-908f-16c2437792ff None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:13.616942 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:13.620078 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 222/438] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:13.621120 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f05f88-4e6d-4938-ba03-638e902454d0 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:13.659302 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d044de2-b7b9-4178-ae3e-91476481bc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:13.659919 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d044de2-b7b9-4178-ae3e-91476481bc6b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:13.660202 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d044de2-b7b9-4178-ae3e-91476481bc6b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:13.660480 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d044de2-b7b9-4178-ae3e-91476481bc6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:13.661203 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d044de2-b7b9-4178-ae3e-91476481bc6b None admin] RBAC: Adding query filter params (group.id=2cfd468a091f4dbba6eaa9d630e9ec6d, role.id=0f58d2782a2840648bbbc2ab26ec02df, scope.project.id=9c9a0023662b41afa1d7bec05fd12eda) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:13.661516 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d044de2-b7b9-4178-ae3e-91476481bc6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:13.663096 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d044de2-b7b9-4178-ae3e-91476481bc6b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:13.668864 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 217/439] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 10 12:23:13 2022] GET /identity/v3/role_assignments?group.id=2cfd468a091f4dbba6eaa9d630e9ec6d&scope.project.id=9c9a0023662b41afa1d7bec05fd12eda&role.id=0f58d2782a2840648bbbc2ab26ec02df => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:13.669653 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d044de2-b7b9-4178-ae3e-91476481bc6b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:13.750802 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f05f88-4e6d-4938-ba03-638e902454d0 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:13.762769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:13.791535 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace81df4-e76a-435f-9466-cd5ac9789ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:13.792321 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace81df4-e76a-435f-9466-cd5ac9789ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:13.792485 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace81df4-e76a-435f-9466-cd5ac9789ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:13.792732 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace81df4-e76a-435f-9466-cd5ac9789ac9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:13.825194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d044de2-b7b9-4178-ae3e-91476481bc6b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:13.874603 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:13.877942 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace81df4-e76a-435f-9466-cd5ac9789ac9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:13.878959 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e3335-3e20-4162-b357-e3742a7a944c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:13.882303 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace81df4-e76a-435f-9466-cd5ac9789ac9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:13.882434 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e3335-3e20-4162-b357-e3742a7a944c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:13.882434 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e3335-3e20-4162-b357-e3742a7a944c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:13.882434 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e3335-3e20-4162-b357-e3742a7a944c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:13.979456 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e3335-3e20-4162-b357-e3742a7a944c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:13.981677 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e3335-3e20-4162-b357-e3742a7a944c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:14.007313 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 223/440] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:23:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:23:14.012179 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace81df4-e76a-435f-9466-cd5ac9789ac9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:14.063457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:14.123250 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 218/441] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:23:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:23:14.129841 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e3335-3e20-4162-b357-e3742a7a944c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:14.136515 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:14.140118 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace81df4-e76a-435f-9466-cd5ac9789ac9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:14.176554 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b177234-5e26-466b-b820-c5bbed6aa77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:14.177189 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b177234-5e26-466b-b820-c5bbed6aa77b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:14.177470 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b177234-5e26-466b-b820-c5bbed6aa77b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:14.177745 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b177234-5e26-466b-b820-c5bbed6aa77b None admin] PATH_INFO: `/v3/projects/b287589e923744ec9ed8d9c9f4196506` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:14.179535 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b177234-5e26-466b-b820-c5bbed6aa77b None admin] RBAC: Authorizing `identity:get_project(project_id=b287589e923744ec9ed8d9c9f4196506)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:14.180957 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b177234-5e26-466b-b820-c5bbed6aa77b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:14.182583 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 224/442] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:23:14 2022] GET /identity/v3/projects/b287589e923744ec9ed8d9c9f4196506 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:14.183706 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0730ac2-e448-40e7-8fdc-49399dd2c8a3 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:14.183986 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0730ac2-e448-40e7-8fdc-49399dd2c8a3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:14.184257 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0730ac2-e448-40e7-8fdc-49399dd2c8a3 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:14.185592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 225/443] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:14.188283 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0730ac2-e448-40e7-8fdc-49399dd2c8a3 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:14.239507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e3335-3e20-4162-b357-e3742a7a944c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:14.273710 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b424be-f550-4460-8313-b79dcb189b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:14.274348 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b424be-f550-4460-8313-b79dcb189b9d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:14.274625 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b424be-f550-4460-8313-b79dcb189b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:14.274895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b424be-f550-4460-8313-b79dcb189b9d None admin] PATH_INFO: `/v3/projects/b287589e923744ec9ed8d9c9f4196506` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:14.276032 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b424be-f550-4460-8313-b79dcb189b9d None admin] RBAC: Authorizing `identity:get_project(project_id=b287589e923744ec9ed8d9c9f4196506)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:14.277464 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b424be-f550-4460-8313-b79dcb189b9d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:14.279390 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 219/444] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:23:14 2022] GET /identity/v3/projects/b287589e923744ec9ed8d9c9f4196506 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:23:14.280434 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645daab-58e2-4efa-891b-c0d123497e15 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:14.280710 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645daab-58e2-4efa-891b-c0d123497e15 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:14.280979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645daab-58e2-4efa-891b-c0d123497e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:14.294431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0730ac2-e448-40e7-8fdc-49399dd2c8a3 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:14.298691 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e645daab-58e2-4efa-891b-c0d123497e15 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:14.299906 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e645daab-58e2-4efa-891b-c0d123497e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:14.320492 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:14.328889 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5b0c8-7bfa-4329-acf3-bd90fa6eb6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:14.331866 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5b0c8-7bfa-4329-acf3-bd90fa6eb6dc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:14.331866 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5b0c8-7bfa-4329-acf3-bd90fa6eb6dc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:14.331866 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5b0c8-7bfa-4329-acf3-bd90fa6eb6dc None admin] PATH_INFO: `/v3/groups/2cfd468a091f4dbba6eaa9d630e9ec6d` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:14.331866 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b5b0c8-7bfa-4329-acf3-bd90fa6eb6dc None admin] RBAC: Authorizing `identity:get_group(group_id=2cfd468a091f4dbba6eaa9d630e9ec6d)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:14.332656 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b5b0c8-7bfa-4329-acf3-bd90fa6eb6dc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:14.334393 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 226/445] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 12:23:14 2022] GET /identity/v3/groups/2cfd468a091f4dbba6eaa9d630e9ec6d => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:23:14.335121 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5b0c8-7bfa-4329-acf3-bd90fa6eb6dc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:14.389102 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 220/446] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:14.389102 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645daab-58e2-4efa-891b-c0d123497e15 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:14.389668 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:14.438258 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5b0c8-7bfa-4329-acf3-bd90fa6eb6dc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:14.465705 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645daab-58e2-4efa-891b-c0d123497e15 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:14.471811 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063b408-e357-45d5-9d39-0407574b9e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:14.472431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063b408-e357-45d5-9d39-0407574b9e1f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:14.472713 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063b408-e357-45d5-9d39-0407574b9e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:14.472983 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063b408-e357-45d5-9d39-0407574b9e1f None admin] PATH_INFO: `/v3/groups/2cfd468a091f4dbba6eaa9d630e9ec6d` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:14.474064 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063b408-e357-45d5-9d39-0407574b9e1f None admin] RBAC: Authorizing `identity:get_group(group_id=2cfd468a091f4dbba6eaa9d630e9ec6d)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:14.475728 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063b408-e357-45d5-9d39-0407574b9e1f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:14.477281 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 227/447] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 12:23:14 2022] GET /identity/v3/groups/2cfd468a091f4dbba6eaa9d630e9ec6d => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:14.478032 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063b408-e357-45d5-9d39-0407574b9e1f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:14.499733 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40eba6e-d2f9-4a9c-8c34-c8f7789df999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:14.500338 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40eba6e-d2f9-4a9c-8c34-c8f7789df999 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:14.500617 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40eba6e-d2f9-4a9c-8c34-c8f7789df999 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:14.500887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40eba6e-d2f9-4a9c-8c34-c8f7789df999 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:14.501598 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40eba6e-d2f9-4a9c-8c34-c8f7789df999 None admin] RBAC: Adding query filter params (group.id=2cfd468a091f4dbba6eaa9d630e9ec6d, role.id=872ba0c4212a4684b539a68fc2c20b25, scope.project.id=b287589e923744ec9ed8d9c9f4196506) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:14.501905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40eba6e-d2f9-4a9c-8c34-c8f7789df999 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:14.503356 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40eba6e-d2f9-4a9c-8c34-c8f7789df999 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:14.509405 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 221/448] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 10 12:23:14 2022] GET /identity/v3/role_assignments?group.id=2cfd468a091f4dbba6eaa9d630e9ec6d&scope.project.id=b287589e923744ec9ed8d9c9f4196506&role.id=872ba0c4212a4684b539a68fc2c20b25 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:14.510167 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40eba6e-d2f9-4a9c-8c34-c8f7789df999 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:14.533890 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:14.589556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063b408-e357-45d5-9d39-0407574b9e1f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:14.593161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:14.627910 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40eba6e-d2f9-4a9c-8c34-c8f7789df999 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:14.628839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e70d45-0ec1-48fe-a426-08ca31788978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:14.629703 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e70d45-0ec1-48fe-a426-08ca31788978 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:14.629988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e70d45-0ec1-48fe-a426-08ca31788978 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:14.630251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e70d45-0ec1-48fe-a426-08ca31788978 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:14.645734 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e70d45-0ec1-48fe-a426-08ca31788978 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:14.650711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-e1e70d45-0ec1-48fe-a426-08ca31788978 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:23:14.650711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:14.650711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:14.650711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:14.650711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:14.650711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:14.650711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:14.650711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:14.650711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:14.650711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:14.650711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:14.650711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:14.650711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:14.650711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:14.650711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:14.650711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:14.651677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:14.651677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:14.651677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:14.651677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:14.651677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:14.651677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:14.651677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:14.651677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:14.651677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:14.651677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:14.651677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:14.651677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:14.651677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:14.651677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:14.651677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:14.651677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:14.652606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:14.652606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:14.652606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:14.652606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:14.652606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:14.652606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:14.652606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:14.652606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:14.652606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:14.652606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:14.652606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:23:14.652606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:14.652606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 228/449] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:23:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:14.657852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e70d45-0ec1-48fe-a426-08ca31788978 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:14.670240 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b12cb-3456-434c-8b41-b6396eba90e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:14.670850 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b12cb-3456-434c-8b41-b6396eba90e5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:14.671128 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b12cb-3456-434c-8b41-b6396eba90e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:14.671464 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b12cb-3456-434c-8b41-b6396eba90e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:14.672192 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b12cb-3456-434c-8b41-b6396eba90e5 None admin] RBAC: Adding query filter params (group.id=2cfd468a091f4dbba6eaa9d630e9ec6d, role.id=0f58d2782a2840648bbbc2ab26ec02df, scope.project.id=b287589e923744ec9ed8d9c9f4196506) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:14.672503 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b12cb-3456-434c-8b41-b6396eba90e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:14.673925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b12cb-3456-434c-8b41-b6396eba90e5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:14.681427 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 222/450] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 10 12:23:14 2022] GET /identity/v3/role_assignments?group.id=2cfd468a091f4dbba6eaa9d630e9ec6d&scope.project.id=b287589e923744ec9ed8d9c9f4196506&role.id=0f58d2782a2840648bbbc2ab26ec02df => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:14.778895 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e70d45-0ec1-48fe-a426-08ca31788978 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:14.817348 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56789809-2e52-4b43-bfd5-de3936b13cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:14.817979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56789809-2e52-4b43-bfd5-de3936b13cbe None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:14.818255 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56789809-2e52-4b43-bfd5-de3936b13cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:14.818532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56789809-2e52-4b43-bfd5-de3936b13cbe None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:14.819614 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56789809-2e52-4b43-bfd5-de3936b13cbe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:14.819922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56789809-2e52-4b43-bfd5-de3936b13cbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:14.821334 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56789809-2e52-4b43-bfd5-de3936b13cbe None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:14.831117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 229/451] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:23:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:14.837284 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b12cb-3456-434c-8b41-b6396eba90e5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:14.930016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:14.945924 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b12cb-3456-434c-8b41-b6396eba90e5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:14.981560 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b64293-59d5-4236-be7d-081c1bd4207c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:14.982174 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b64293-59d5-4236-be7d-081c1bd4207c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:14.982458 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b64293-59d5-4236-be7d-081c1bd4207c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:14.982734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b64293-59d5-4236-be7d-081c1bd4207c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:15.051591 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b64293-59d5-4236-be7d-081c1bd4207c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:15.053091 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b64293-59d5-4236-be7d-081c1bd4207c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:15.132594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:15.148198 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 223/452] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:23:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:23:15.153841 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56789809-2e52-4b43-bfd5-de3936b13cbe None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:15.181023 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:15.250445 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56789809-2e52-4b43-bfd5-de3936b13cbe None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:15.284546 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31722bd-d8b4-4ee6-bb41-8bdffe6e2b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:15.285169 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31722bd-d8b4-4ee6-bb41-8bdffe6e2b70 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:15.285444 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31722bd-d8b4-4ee6-bb41-8bdffe6e2b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:15.285712 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31722bd-d8b4-4ee6-bb41-8bdffe6e2b70 None admin] PATH_INFO: `/v3/projects/3c65e78604254a1dbcd377669ee26423` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:15.286810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31722bd-d8b4-4ee6-bb41-8bdffe6e2b70 None admin] RBAC: Authorizing `identity:get_project(project_id=3c65e78604254a1dbcd377669ee26423)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:15.288307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31722bd-d8b4-4ee6-bb41-8bdffe6e2b70 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:15.289899 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 230/453] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:23:15 2022] GET /identity/v3/projects/3c65e78604254a1dbcd377669ee26423 => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:15.295317 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b64293-59d5-4236-be7d-081c1bd4207c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:15.391584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b64293-59d5-4236-be7d-081c1bd4207c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:15.423916 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab3def-ee6a-4e64-83cf-89a7612a1e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:15.424541 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab3def-ee6a-4e64-83cf-89a7612a1e8c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:15.424823 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab3def-ee6a-4e64-83cf-89a7612a1e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:15.425096 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab3def-ee6a-4e64-83cf-89a7612a1e8c None admin] PATH_INFO: `/v3/groups/e87afcd2af4b484e88e1debe318182c1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:15.426202 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab3def-ee6a-4e64-83cf-89a7612a1e8c None admin] RBAC: Authorizing `identity:get_group(group_id=e87afcd2af4b484e88e1debe318182c1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:15.427689 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab3def-ee6a-4e64-83cf-89a7612a1e8c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:15.429986 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 224/454] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 12:23:15 2022] GET /identity/v3/groups/e87afcd2af4b484e88e1debe318182c1 => generated 229 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:15.438208 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31722bd-d8b4-4ee6-bb41-8bdffe6e2b70 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:15.528627 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:15.534943 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31722bd-d8b4-4ee6-bb41-8bdffe6e2b70 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:15.567678 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f5e50-c401-4ab9-90ef-00e381b6c4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:15.568271 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f5e50-c401-4ab9-90ef-00e381b6c4d6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:15.569137 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f5e50-c401-4ab9-90ef-00e381b6c4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:15.569468 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f5e50-c401-4ab9-90ef-00e381b6c4d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:15.570191 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f5e50-c401-4ab9-90ef-00e381b6c4d6 None admin] RBAC: Adding query filter params (group.id=e87afcd2af4b484e88e1debe318182c1, role.id=bffbe227cda1418cbb58394ddff6f861, scope.project.id=3c65e78604254a1dbcd377669ee26423) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:15.570503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f5e50-c401-4ab9-90ef-00e381b6c4d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:15.571949 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f5e50-c401-4ab9-90ef-00e381b6c4d6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:15.577482 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 231/455] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 10 12:23:15 2022] GET /identity/v3/role_assignments?group.id=e87afcd2af4b484e88e1debe318182c1&scope.project.id=3c65e78604254a1dbcd377669ee26423&role.id=bffbe227cda1418cbb58394ddff6f861 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:17.970794 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcbebe-1715-4c01-acbf-75cc0d1ee701 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:17.971869 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcbebe-1715-4c01-acbf-75cc0d1ee701 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:17.972352 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcbebe-1715-4c01-acbf-75cc0d1ee701 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:17.974207 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 225/456] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:17.980572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1c450-65d0-462b-ad72-c832036104d2 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:17.981015 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1c450-65d0-462b-ad72-c832036104d2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:17.981111 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1c450-65d0-462b-ad72-c832036104d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:18.008578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a8b1c450-65d0-462b-ad72-c832036104d2 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:18.009793 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a8b1c450-65d0-462b-ad72-c832036104d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:18.102667 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 232/457] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:18.152802 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dcbebe-1715-4c01-acbf-75cc0d1ee701 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:18.221704 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:18.247241 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbf8cd-482d-4fb1-bc2d-e0a4c1afe3e7 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:18.247846 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbf8cd-482d-4fb1-bc2d-e0a4c1afe3e7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:18.248332 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbf8cd-482d-4fb1-bc2d-e0a4c1afe3e7 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:18.250307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 233/458] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:18.255162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89b683-ca6d-4cf8-8999-842a8c2efc19 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:18.255734 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89b683-ca6d-4cf8-8999-842a8c2efc19 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:18.256903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89b683-ca6d-4cf8-8999-842a8c2efc19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:18.300968 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7b89b683-ca6d-4cf8-8999-842a8c2efc19 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:18.302333 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7b89b683-ca6d-4cf8-8999-842a8c2efc19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:18.351757 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dcbebe-1715-4c01-acbf-75cc0d1ee701 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:18.423396 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792a48d-3608-4dab-a9c1-87769f84175b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:18.424029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792a48d-3608-4dab-a9c1-87769f84175b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:18.424305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792a48d-3608-4dab-a9c1-87769f84175b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:18.424575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792a48d-3608-4dab-a9c1-87769f84175b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:18.425344 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 234/459] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:18.436241 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1792a48d-3608-4dab-a9c1-87769f84175b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:18.437870 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1792a48d-3608-4dab-a9c1-87769f84175b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:18.453429 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-1792a48d-3608-4dab-a9c1-87769f84175b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:18.453429 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:18.453429 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:18.453429 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:18.453429 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:18.453429 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:18.453429 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:18.453429 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:18.453429 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:18.453429 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:18.453429 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:18.453429 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:18.453429 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:18.453429 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:18.453429 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:18.453429 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:18.454601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:18.454601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:18.454601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:18.454601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:18.454601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:18.454601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:18.454601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:18.454601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:18.454601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:18.454601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:18.454601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:18.454601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:18.454601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:18.454601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:18.454601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:18.454601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:18.454601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:18.455671 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:18.455671 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:18.455671 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:18.457595 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 226/460] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:18.464089 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b89b683-ca6d-4cf8-8999-842a8c2efc19 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:18.468182 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792a48d-3608-4dab-a9c1-87769f84175b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:18.486290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:18.593793 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b89b683-ca6d-4cf8-8999-842a8c2efc19 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:18.618804 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792a48d-3608-4dab-a9c1-87769f84175b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:18.622123 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:18.637628 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288022b-a97f-4bb6-a165-4eea37721ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:18.638539 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288022b-a97f-4bb6-a165-4eea37721ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:18.638995 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288022b-a97f-4bb6-a165-4eea37721ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:18.643168 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288022b-a97f-4bb6-a165-4eea37721ec4 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:18.643168 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7288022b-a97f-4bb6-a165-4eea37721ec4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:18.643168 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7288022b-a97f-4bb6-a165-4eea37721ec4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:18.643168 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7288022b-a97f-4bb6-a165-4eea37721ec4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:18.656210 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0c98e-6e4f-4df6-85a8-0b1079f05720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:18.657079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0c98e-6e4f-4df6-85a8-0b1079f05720 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:18.657557 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0c98e-6e4f-4df6-85a8-0b1079f05720 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:18.658070 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0c98e-6e4f-4df6-85a8-0b1079f05720 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:18.663406 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 235/461] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:18.664896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1906f89-592c-465b-aca4-11d1ebc6f315 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:18.665466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1906f89-592c-465b-aca4-11d1ebc6f315 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:18.665934 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1906f89-592c-465b-aca4-11d1ebc6f315 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:18.667618 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 236/462] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:18.672517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ec22f-9e00-4aec-8710-8804ca8faf84 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:18.673080 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ec22f-9e00-4aec-8710-8804ca8faf84 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:18.673572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ec22f-9e00-4aec-8710-8804ca8faf84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:18.675056 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f0c98e-6e4f-4df6-85a8-0b1079f05720 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:18.676639 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f0c98e-6e4f-4df6-85a8-0b1079f05720 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:18.694310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-e5f0c98e-6e4f-4df6-85a8-0b1079f05720 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:18.694310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:18.694310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:18.694310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:18.694310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:18.694310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:18.694310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:18.694310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:18.694310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:18.694310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:18.694310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:18.694310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:18.694310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:18.694310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:18.694310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:18.694310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:18.695650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:18.695650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:18.695650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:18.695650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:18.695650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:18.695650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:18.695650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:18.695650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:18.695650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:18.695650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:18.695650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:18.695650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:18.695650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:18.695650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:18.695650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:18.695650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:18.695650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:18.696566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:18.696566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:18.696566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:18.698657 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 227/463] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:18.699688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0c98e-6e4f-4df6-85a8-0b1079f05720 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:18.704160 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ce8ec22f-9e00-4aec-8710-8804ca8faf84 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:18.705359 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ce8ec22f-9e00-4aec-8710-8804ca8faf84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:18.776924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 237/464] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:18.778182 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979996f-286f-4307-8f5d-e278cf67d89c None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:18.778883 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979996f-286f-4307-8f5d-e278cf67d89c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:18.779160 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979996f-286f-4307-8f5d-e278cf67d89c None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:18.780984 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 238/465] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:18.781839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979996f-286f-4307-8f5d-e278cf67d89c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:18.795219 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:18.808828 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0c98e-6e4f-4df6-85a8-0b1079f05720 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:18.846436 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d55b33-aefb-43f6-a0f0-bcbaf0346062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:18.847041 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d55b33-aefb-43f6-a0f0-bcbaf0346062 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:18.847378 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d55b33-aefb-43f6-a0f0-bcbaf0346062 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:18.847644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d55b33-aefb-43f6-a0f0-bcbaf0346062 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:18.848277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d55b33-aefb-43f6-a0f0-bcbaf0346062 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:18.849696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d55b33-aefb-43f6-a0f0-bcbaf0346062 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:18.894584 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979996f-286f-4307-8f5d-e278cf67d89c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:18.930157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d54dfe-7acf-43e9-8ec5-cc5145bc805d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:18.930759 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d54dfe-7acf-43e9-8ec5-cc5145bc805d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:18.931036 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d54dfe-7acf-43e9-8ec5-cc5145bc805d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:18.931339 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d54dfe-7acf-43e9-8ec5-cc5145bc805d None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:18.931979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d54dfe-7acf-43e9-8ec5-cc5145bc805d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:18.932284 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d54dfe-7acf-43e9-8ec5-cc5145bc805d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:18.933637 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d54dfe-7acf-43e9-8ec5-cc5145bc805d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:18.949654 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 239/466] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:18.949654 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa0ebb-4111-4387-a997-35422a843bde None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:18.949654 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa0ebb-4111-4387-a997-35422a843bde None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:18.949654 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa0ebb-4111-4387-a997-35422a843bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:18.956014 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 228/467] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 12:23:18 2022] POST /identity/v3/users => generated 246 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:23:18.956693 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d55b33-aefb-43f6-a0f0-bcbaf0346062 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:18.982160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:18.991771 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-cffa0ebb-4111-4387-a997-35422a843bde None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:18.993034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cffa0ebb-4111-4387-a997-35422a843bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:19.081406 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:19.095205 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 240/468] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:19.095205 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa0ebb-4111-4387-a997-35422a843bde None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:19.107557 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d55b33-aefb-43f6-a0f0-bcbaf0346062 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:19.167194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce5887-2b8d-48b7-9d53-012be77306fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:19.167873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce5887-2b8d-48b7-9d53-012be77306fb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:19.168831 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce5887-2b8d-48b7-9d53-012be77306fb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:19.168831 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce5887-2b8d-48b7-9d53-012be77306fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:19.180180 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce5887-2b8d-48b7-9d53-012be77306fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:19.181632 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce5887-2b8d-48b7-9d53-012be77306fb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:19.196460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-83ce5887-2b8d-48b7-9d53-012be77306fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:19.196460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:19.196460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:19.196460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:19.196460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:19.196460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:19.196460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:19.196460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:19.196460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:19.196460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:19.196460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:19.196460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:19.196460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:19.196460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:19.196460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:19.196460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:19.197391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:19.197391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:19.197391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:19.197391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:19.197391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:19.197391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:19.197391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:19.197391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:19.197391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:19.197391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:19.197391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:19.197391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:19.197391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:19.197391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:19.197391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:19.197391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:19.197391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:19.198247 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:19.198247 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:19.198247 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:19.198247 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 229/469] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:19.198474 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce5887-2b8d-48b7-9d53-012be77306fb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:19.221323 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa0ebb-4111-4387-a997-35422a843bde None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:19.264036 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e75a3a-3560-404a-86a6-91729a5ba7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:19.264036 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e75a3a-3560-404a-86a6-91729a5ba7d3 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:19.264036 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e75a3a-3560-404a-86a6-91729a5ba7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:19.264036 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e75a3a-3560-404a-86a6-91729a5ba7d3 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:19.264420 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e75a3a-3560-404a-86a6-91729a5ba7d3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:19.265684 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e75a3a-3560-404a-86a6-91729a5ba7d3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:19.273346 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:19.317128 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce5887-2b8d-48b7-9d53-012be77306fb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:19.354021 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0165e4b-48f2-42a7-8179-3daa714de434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:19.354578 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0165e4b-48f2-42a7-8179-3daa714de434 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:19.354859 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0165e4b-48f2-42a7-8179-3daa714de434 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:19.355130 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0165e4b-48f2-42a7-8179-3daa714de434 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:19.358656 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 241/470] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 12:23:19 2022] POST /identity/v3/users => generated 248 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:23:19.359466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e75a3a-3560-404a-86a6-91729a5ba7d3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:19.386350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0165e4b-48f2-42a7-8179-3daa714de434 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:19.390213 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0165e4b-48f2-42a7-8179-3daa714de434 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:19.420811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-e0165e4b-48f2-42a7-8179-3daa714de434 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:19.420811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:19.420811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:19.420811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:19.420811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:19.420811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:19.420811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:19.420811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:19.420811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:19.420811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:19.420811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:19.420811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:19.420811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:19.420811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:19.420811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:19.420811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:19.421796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:19.421796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:19.421796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:19.421796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:19.421796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:19.421796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:19.421796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:19.421796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:19.421796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:19.421796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:19.421796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:19.421796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:19.421796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:19.421796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:19.421796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:19.421796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:19.421796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:19.422646 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:19.422646 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:19.422646 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:19.422646 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 230/471] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:19.429628 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0165e4b-48f2-42a7-8179-3daa714de434 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:19.483318 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:19.545239 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e75a3a-3560-404a-86a6-91729a5ba7d3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:19.572626 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0165e4b-48f2-42a7-8179-3daa714de434 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:19.594651 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:19.595013 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031d608-1358-48af-a7e8-553f64b77dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:19.596014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031d608-1358-48af-a7e8-553f64b77dad None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:19.596292 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031d608-1358-48af-a7e8-553f64b77dad None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:19.596565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031d608-1358-48af-a7e8-553f64b77dad None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:19.597217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1031d608-1358-48af-a7e8-553f64b77dad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:19.597519 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1031d608-1358-48af-a7e8-553f64b77dad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:19.600494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1031d608-1358-48af-a7e8-553f64b77dad None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:19.617225 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0370e9c-4957-420e-a279-b40bd71c0129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:19.617405 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 242/472] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:19.618069 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0370e9c-4957-420e-a279-b40bd71c0129 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:19.618347 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0370e9c-4957-420e-a279-b40bd71c0129 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:19.618619 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0370e9c-4957-420e-a279-b40bd71c0129 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:19.618736 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90ac0d-4e33-4525-a4e0-930d298b77e5 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:19.618982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90ac0d-4e33-4525-a4e0-930d298b77e5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:19.619280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90ac0d-4e33-4525-a4e0-930d298b77e5 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:19.619444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0370e9c-4957-420e-a279-b40bd71c0129 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:19.619720 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0370e9c-4957-420e-a279-b40bd71c0129 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:19.620854 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 243/473] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:19.621410 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0370e9c-4957-420e-a279-b40bd71c0129 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:19.625537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90ac0d-4e33-4525-a4e0-930d298b77e5 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:19.637708 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 231/474] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:19.639720 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500d2b2-e8ca-4e42-875e-79d933a97a39 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:19.640006 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500d2b2-e8ca-4e42-875e-79d933a97a39 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:19.640272 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500d2b2-e8ca-4e42-875e-79d933a97a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:19.665326 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b500d2b2-e8ca-4e42-875e-79d933a97a39 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:19.666592 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b500d2b2-e8ca-4e42-875e-79d933a97a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:19.730967 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90ac0d-4e33-4525-a4e0-930d298b77e5 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:19.759491 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 232/475] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:19.759954 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:19.760694 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500d2b2-e8ca-4e42-875e-79d933a97a39 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:19.783275 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86ede3-673a-4e99-bfc6-f5a9f5bdff96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:19.784037 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86ede3-673a-4e99-bfc6-f5a9f5bdff96 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:19.784317 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86ede3-673a-4e99-bfc6-f5a9f5bdff96 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:19.784643 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86ede3-673a-4e99-bfc6-f5a9f5bdff96 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:19.785342 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86ede3-673a-4e99-bfc6-f5a9f5bdff96 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:19.787023 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86ede3-673a-4e99-bfc6-f5a9f5bdff96 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:19.874917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500d2b2-e8ca-4e42-875e-79d933a97a39 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:19.884647 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:19.913927 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3b2f7-1a06-4186-be18-0c0ec29a63b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:19.914601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3b2f7-1a06-4186-be18-0c0ec29a63b7 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:19.914883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3b2f7-1a06-4186-be18-0c0ec29a63b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:19.915155 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3b2f7-1a06-4186-be18-0c0ec29a63b7 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:19.915888 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3b2f7-1a06-4186-be18-0c0ec29a63b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:19.918310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3b2f7-1a06-4186-be18-0c0ec29a63b7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:19.920485 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 244/476] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 12:23:19 2022] POST /identity/v3/users => generated 249 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:23:19.921973 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86ede3-673a-4e99-bfc6-f5a9f5bdff96 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:20.033174 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 233/477] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 12:23:19 2022] POST /identity/v3/users => generated 248 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:23:20.059895 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86ede3-673a-4e99-bfc6-f5a9f5bdff96 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:20.101582 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8dd8f-3793-43da-9338-c3b382cbd768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:20.102215 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8dd8f-3793-43da-9338-c3b382cbd768 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:20.102499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8dd8f-3793-43da-9338-c3b382cbd768 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:20.102775 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8dd8f-3793-43da-9338-c3b382cbd768 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:20.103485 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8dd8f-3793-43da-9338-c3b382cbd768 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:20.104908 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8dd8f-3793-43da-9338-c3b382cbd768 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:20.143833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.sql.core [None req-efd8dd8f-3793-43da-9338-c3b382cbd768 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 12:23:20.143833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 10 12:23:20.143833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [parameters: {'id': '94cf8b51ff064eae9c48a267302dfb32', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 10 12:23:20.143833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80202) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 12:23:20.162762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-efd8dd8f-3793-43da-9338-c3b382cbd768 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 10 12:23:20.162762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:20.162762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 12:23:20.162762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 12:23:20.162762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 12:23:20.162762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 12:23:20.162762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 12:23:20.162762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 12:23:20.162762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 12:23:20.162762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application conn.query(q) Aug 10 12:23:20.162762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 12:23:20.162762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 12:23:20.162762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 12:23:20.162762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application result.read() Aug 10 12:23:20.162762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 12:23:20.163939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 12:23:20.163939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 12:23:20.163939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 12:23:20.163939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 12:23:20.163939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 12:23:20.163939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 12:23:20.163939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 12:23:20.163939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 12:23:20.163939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:20.163939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 12:23:20.163939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:20.163939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:20.163939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 12:23:20.163939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 12:23:20.163939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 10 12:23:20.163939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return ref.to_dict() Aug 10 12:23:20.163939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 12:23:20.163939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application next(self.gen) Aug 10 12:23:20.164839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 12:23:20.164839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application yield resource Aug 10 12:23:20.164839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 12:23:20.164839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application next(self.gen) Aug 10 12:23:20.164839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 12:23:20.164839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 12:23:20.164839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 12:23:20.164839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 12:23:20.164839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 12:23:20.164839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise self.value Aug 10 12:23:20.164839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 12:23:20.164839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 12:23:20.164839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 12:23:20.164839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application session.commit() Aug 10 12:23:20.164839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 12:23:20.164839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 12:23:20.164839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 12:23:20.165693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 12:23:20.165693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 12:23:20.165693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self.session.flush() Aug 10 12:23:20.165693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 12:23:20.165693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 12:23:20.165693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 12:23:20.165693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 12:23:20.165693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 12:23:20.165693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application compat.raise_( Aug 10 12:23:20.165693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 12:23:20.165693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception Aug 10 12:23:20.165693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 12:23:20.165693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 12:23:20.165693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 12:23:20.165693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 12:23:20.165693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 12:23:20.165693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 12:23:20.166552 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 12:23:20.166552 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 12:23:20.166552 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 10 12:23:20.166552 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 10 12:23:20.166552 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 12:23:20.166552 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 12:23:20.166552 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 12:23:20.166552 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 12:23:20.166552 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 12:23:20.166552 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 12:23:20.166552 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 12:23:20.166552 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 12:23:20.166552 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 12:23:20.166552 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 12:23:20.166552 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 12:23:20.166552 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception Aug 10 12:23:20.167385 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 12:23:20.167385 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 12:23:20.167385 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 12:23:20.167385 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 12:23:20.167385 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 12:23:20.167385 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 12:23:20.167385 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 12:23:20.167385 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application conn.query(q) Aug 10 12:23:20.167385 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 12:23:20.167385 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 12:23:20.167385 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 12:23:20.167385 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application result.read() Aug 10 12:23:20.167385 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 12:23:20.167385 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 12:23:20.167385 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 12:23:20.167385 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 12:23:20.169593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 12:23:20.169593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 12:23:20.169593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 12:23:20.169593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 12:23:20.169593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 12:23:20.169593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 10 12:23:20.169593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [parameters: {'id': '94cf8b51ff064eae9c48a267302dfb32', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 10 12:23:20.169593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 12:23:20.169593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:20.169593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:23:20.169593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:20.169593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:20.169593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:20.169593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:20.169593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:20.169593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:20.171633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:20.171633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:20.171633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:20.171633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:20.171633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:20.171633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:20.171633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 10 12:23:20.171633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 10 12:23:20.171633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:20.171633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:20.171633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 10 12:23:20.171633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 10 12:23:20.171633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 12:23:20.171633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 12:23:20.171633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 10 12:23:20.171633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:20.173594 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 245/478] 10.222.0.49 () {60 vars in 1131 bytes} [Wed Aug 10 12:23:19 2022] POST /identity/v3/roles => generated 144 bytes in 244 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:20.173729 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3b2f7-1a06-4186-be18-0c0ec29a63b7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:20.299608 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3b2f7-1a06-4186-be18-0c0ec29a63b7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:20.337955 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a204401-c7fd-4b84-b332-9d37390baa79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:20.338594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a204401-c7fd-4b84-b332-9d37390baa79 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:20.338874 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a204401-c7fd-4b84-b332-9d37390baa79 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:20.339142 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a204401-c7fd-4b84-b332-9d37390baa79 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:20.350786 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a204401-c7fd-4b84-b332-9d37390baa79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:20.356251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-3a204401-c7fd-4b84-b332-9d37390baa79 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:23:20.356251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:20.356251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:20.356251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:20.356251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:20.356251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:20.356251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:20.356251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:20.356251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:20.356251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:20.356251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:20.356251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:20.356251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:20.356251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:20.356251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:20.356251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:20.357201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:20.357201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:20.357201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:20.357201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:20.357201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:20.357201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:20.357201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:20.357201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:20.357201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:20.357201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:20.357201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:20.357201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:20.357201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:20.357201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:20.357201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:20.357201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:20.358226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:20.358226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:20.358226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:20.358226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:20.358226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:20.358226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:20.358226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:20.358226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:20.358226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:20.358226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:20.358226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:23:20.358226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:20.358226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 234/479] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 12:23:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:20.363976 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8dd8f-3793-43da-9338-c3b382cbd768 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:20.409563 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:20.501297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8dd8f-3793-43da-9338-c3b382cbd768 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:20.543500 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df7dfa-c553-4362-9f86-d1a30fc92b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:20.544156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df7dfa-c553-4362-9f86-d1a30fc92b8a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:20.544403 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df7dfa-c553-4362-9f86-d1a30fc92b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:20.544732 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df7dfa-c553-4362-9f86-d1a30fc92b8a None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:20.545404 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df7dfa-c553-4362-9f86-d1a30fc92b8a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:20.545706 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df7dfa-c553-4362-9f86-d1a30fc92b8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:20.547058 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df7dfa-c553-4362-9f86-d1a30fc92b8a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:20.559445 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 246/480] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:21.474717 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89573afa-7fe5-4833-969c-96093e936aa2 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:21.476360 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89573afa-7fe5-4833-969c-96093e936aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:21.476872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89573afa-7fe5-4833-969c-96093e936aa2 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:21.478675 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 235/481] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:21.498253 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441fa83-bc6d-4e99-aec3-00d5ba774810 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:21.499170 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441fa83-bc6d-4e99-aec3-00d5ba774810 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:21.499723 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441fa83-bc6d-4e99-aec3-00d5ba774810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:21.529231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-4441fa83-bc6d-4e99-aec3-00d5ba774810 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:21.530805 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4441fa83-bc6d-4e99-aec3-00d5ba774810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:21.625269 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 247/482] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:21.654835 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89573afa-7fe5-4833-969c-96093e936aa2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:21.685593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e018b-3f42-4d7b-9137-baf06d8c5516 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:21.686233 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e018b-3f42-4d7b-9137-baf06d8c5516 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:21.686702 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e018b-3f42-4d7b-9137-baf06d8c5516 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:21.688503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 248/483] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:23:21.689869 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:21.694665 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071c5ed-dbc2-461e-8819-a7c8ceb5192d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:21.695363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071c5ed-dbc2-461e-8819-a7c8ceb5192d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:21.695835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071c5ed-dbc2-461e-8819-a7c8ceb5192d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:21.715353 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a071c5ed-dbc2-461e-8819-a7c8ceb5192d None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:21.716847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a071c5ed-dbc2-461e-8819-a7c8ceb5192d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:21.747105 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:21.814017 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 249/484] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:21.832809 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89573afa-7fe5-4833-969c-96093e936aa2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:21.840275 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a071c5ed-dbc2-461e-8819-a7c8ceb5192d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:21.884943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b1c3d-1284-4ebb-9f68-302059024f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:21.885579 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b1c3d-1284-4ebb-9f68-302059024f5c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:21.885853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b1c3d-1284-4ebb-9f68-302059024f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:21.886120 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b1c3d-1284-4ebb-9f68-302059024f5c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:21.908460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b1c3d-1284-4ebb-9f68-302059024f5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:21.913736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a77b1c3d-1284-4ebb-9f68-302059024f5c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:21.913736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:21.913736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:21.913736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:21.913736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:21.913736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:21.913736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:21.913736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:21.913736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:21.913736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:21.913736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:21.913736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:21.913736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:21.913736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:21.913736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:21.913736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:21.914709 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:21.914709 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:21.914709 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:21.914709 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:21.914709 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:21.914709 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:21.914709 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:21.914709 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:21.914709 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:21.914709 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:21.914709 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:21.914709 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:21.914709 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:21.914709 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:21.914709 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:21.914709 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:21.915575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:21.915575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:21.915575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:21.915575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:21.915575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:21.915575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:21.915575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:21.915575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:21.915575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:21.915575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:21.915575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:21.915575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:21.923639 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 236/485] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 12:23:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:21.930075 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b1c3d-1284-4ebb-9f68-302059024f5c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:21.965835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a071c5ed-dbc2-461e-8819-a7c8ceb5192d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:22.007545 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac39ca-da11-49f6-811e-5d9aa07b65f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:22.008633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac39ca-da11-49f6-811e-5d9aa07b65f0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:22.009132 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac39ca-da11-49f6-811e-5d9aa07b65f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:22.009663 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac39ca-da11-49f6-811e-5d9aa07b65f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:22.021589 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ac39ca-da11-49f6-811e-5d9aa07b65f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:22.027290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-e3ac39ca-da11-49f6-811e-5d9aa07b65f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:22.027290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:22.027290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:22.027290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:22.027290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:22.027290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:22.027290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:22.027290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:22.027290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:22.027290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:22.027290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:22.027290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:22.027290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:22.027290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:22.027290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:22.027290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:22.028187 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:22.028187 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:22.028187 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:22.028187 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:22.028187 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:22.028187 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:22.028187 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:22.028187 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:22.028187 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:22.028187 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:22.028187 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:22.028187 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:22.028187 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:22.028187 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:22.028187 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:22.028187 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:22.028988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:22.028988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:22.028988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:22.028988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:22.028988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:22.028988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:22.028988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:22.028988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:22.028988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:22.028988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:22.028988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:22.028988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:22.031980 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 250/486] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 12:23:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:23:22.039375 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac39ca-da11-49f6-811e-5d9aa07b65f0 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:22.060730 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:22.091008 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:22.137170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b1c3d-1284-4ebb-9f68-302059024f5c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:22.157094 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac39ca-da11-49f6-811e-5d9aa07b65f0 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:22.196343 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec5db3c-5a69-4ee8-8542-57b8633d240c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:22.197330 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5db3c-5a69-4ee8-8542-57b8633d240c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:22.197815 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5db3c-5a69-4ee8-8542-57b8633d240c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:22.198304 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5db3c-5a69-4ee8-8542-57b8633d240c None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:22.199198 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec5db3c-5a69-4ee8-8542-57b8633d240c None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:22.199705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec5db3c-5a69-4ee8-8542-57b8633d240c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:22.201413 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec5db3c-5a69-4ee8-8542-57b8633d240c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:22.212314 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 251/487] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 12:23:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:23:22.218164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec5db3c-5a69-4ee8-8542-57b8633d240c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:22.218619 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e549a-ec35-4ec2-a58c-db89dc736ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:22.219372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e549a-ec35-4ec2-a58c-db89dc736ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:22.219664 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e549a-ec35-4ec2-a58c-db89dc736ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:22.219927 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e549a-ec35-4ec2-a58c-db89dc736ce4 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:22.220593 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e549a-ec35-4ec2-a58c-db89dc736ce4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:22.220896 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e549a-ec35-4ec2-a58c-db89dc736ce4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:22.222262 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e549a-ec35-4ec2-a58c-db89dc736ce4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:22.242567 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 237/488] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 12:23:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:23:22.247973 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e549a-ec35-4ec2-a58c-db89dc736ce4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:22.310884 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:22.336287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec5db3c-5a69-4ee8-8542-57b8633d240c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:22.374487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb88a89-175b-41c6-96f7-d5d1b5239f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:22.375037 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb88a89-175b-41c6-96f7-d5d1b5239f48 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:22.375361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb88a89-175b-41c6-96f7-d5d1b5239f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:22.375636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb88a89-175b-41c6-96f7-d5d1b5239f48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:22.387567 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb88a89-175b-41c6-96f7-d5d1b5239f48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:22.389114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb88a89-175b-41c6-96f7-d5d1b5239f48 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:22.406074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-7cb88a89-175b-41c6-96f7-d5d1b5239f48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:22.406074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:22.406074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:22.406074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:22.406074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:22.406074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:22.406074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:22.406074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:22.406074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:22.406074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:22.406074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:22.406074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:22.406074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:22.406074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:22.406074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:22.406074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:22.407014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:22.407014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:22.407014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:22.407014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:22.407014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:22.407014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:22.407014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:22.407014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:22.407014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:22.407014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:22.407014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:22.407014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:22.407014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:22.407014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:22.407014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:22.407014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:22.407014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:22.408131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:22.408131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:22.408131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:22.408131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 252/489] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:23:22.408879 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6046d-a361-47e5-be9e-fbe28b1eec10 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:22.408992 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6046d-a361-47e5-be9e-fbe28b1eec10 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:22.411253 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6046d-a361-47e5-be9e-fbe28b1eec10 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:22.415584 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 253/490] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:22.416374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6046d-a361-47e5-be9e-fbe28b1eec10 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:22.424619 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e549a-ec35-4ec2-a58c-db89dc736ce4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:22.461389 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415c5af-ff84-4511-8930-9f19f19d07cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:22.462016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415c5af-ff84-4511-8930-9f19f19d07cc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:22.462292 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415c5af-ff84-4511-8930-9f19f19d07cc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:22.462615 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415c5af-ff84-4511-8930-9f19f19d07cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:22.473902 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415c5af-ff84-4511-8930-9f19f19d07cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:22.476200 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415c5af-ff84-4511-8930-9f19f19d07cc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:22.479208 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:22.492466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-0415c5af-ff84-4511-8930-9f19f19d07cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:22.492466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:22.492466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:22.492466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:22.492466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:22.492466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:22.492466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:22.492466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:22.492466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:22.492466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:22.492466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:22.492466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:22.492466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:22.492466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:22.492466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:22.492466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:22.496379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:22.496379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:22.496379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:22.496379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:22.496379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:22.496379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:22.496379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:22.496379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:22.496379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:22.496379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:22.496379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:22.496379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:22.496379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:22.496379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:22.496379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:22.496379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:22.496379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:22.498547 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:22.498547 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:22.498547 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:22.498547 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 238/491] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:23:22.498547 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f4887-ed48-4285-a7e6-a95b1e081808 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:22.498547 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f4887-ed48-4285-a7e6-a95b1e081808 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:22.498547 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f4887-ed48-4285-a7e6-a95b1e081808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:22.516799 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-770f4887-ed48-4285-a7e6-a95b1e081808 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:22.518076 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-770f4887-ed48-4285-a7e6-a95b1e081808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:22.550981 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6046d-a361-47e5-be9e-fbe28b1eec10 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:22.581169 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:22.588465 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec079e7-97d5-44af-a6a1-33265c4616b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:22.593095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec079e7-97d5-44af-a6a1-33265c4616b3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:22.593095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec079e7-97d5-44af-a6a1-33265c4616b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:22.593095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec079e7-97d5-44af-a6a1-33265c4616b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:22.593095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec079e7-97d5-44af-a6a1-33265c4616b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:22.593095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec079e7-97d5-44af-a6a1-33265c4616b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:22.593095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec079e7-97d5-44af-a6a1-33265c4616b3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:22.597161 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 239/492] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:22.597921 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f4887-ed48-4285-a7e6-a95b1e081808 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:22.620899 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 254/493] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:22.620899 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89181727-8a1c-4f53-8346-a692b9943598 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:22.620899 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89181727-8a1c-4f53-8346-a692b9943598 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:22.620899 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89181727-8a1c-4f53-8346-a692b9943598 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:22.620899 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 255/494] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:22.621815 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89181727-8a1c-4f53-8346-a692b9943598 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:22.723078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f4887-ed48-4285-a7e6-a95b1e081808 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:22.735578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89181727-8a1c-4f53-8346-a692b9943598 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:22.750947 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:22.760935 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f775218-6b46-4e38-b2c4-38e306ca1dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:22.761586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f775218-6b46-4e38-b2c4-38e306ca1dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:22.761868 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f775218-6b46-4e38-b2c4-38e306ca1dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:22.762086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f775218-6b46-4e38-b2c4-38e306ca1dd6 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:22.762715 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f775218-6b46-4e38-b2c4-38e306ca1dd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:22.763016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f775218-6b46-4e38-b2c4-38e306ca1dd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:22.764476 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f775218-6b46-4e38-b2c4-38e306ca1dd6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:22.771667 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865151d7-a03c-461a-93f0-a75f88b5f515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:22.772865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865151d7-a03c-461a-93f0-a75f88b5f515 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:22.773146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865151d7-a03c-461a-93f0-a75f88b5f515 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:22.773428 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865151d7-a03c-461a-93f0-a75f88b5f515 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:22.776479 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 240/495] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:22.777450 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f775218-6b46-4e38-b2c4-38e306ca1dd6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:22.780950 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865151d7-a03c-461a-93f0-a75f88b5f515 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:22.782405 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865151d7-a03c-461a-93f0-a75f88b5f515 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:22.795642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-865151d7-a03c-461a-93f0-a75f88b5f515 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 12:23:22.795642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:22.795642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:22.795642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:22.795642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:22.795642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:22.795642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:22.795642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:22.795642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:22.795642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:22.795642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:22.795642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:22.795642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:23:22.795642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:23:22.795642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:23:22.795642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:23:22.796577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:22.796577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:22.796577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:23:22.796577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:22.796577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:23:22.796577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:22.796577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:22.796577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:22.796577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:22.796577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:22.796577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:22.796577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:22.796577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:22.796577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:22.796577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:22.796577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:22.796577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:22.796577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:22.797456 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:22.797456 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:22.797456 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:23:22.797456 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:23:22.797456 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:23:22.797456 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:23:22.797456 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:23:22.797456 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:23:22.797456 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 12:23:22.797456 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:22.797456 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 256/496] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:23:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:22.798018 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1cd52-15b3-4a2d-8b5e-c45ba0e8cb09 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:22.798700 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1cd52-15b3-4a2d-8b5e-c45ba0e8cb09 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:22.798700 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1cd52-15b3-4a2d-8b5e-c45ba0e8cb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:22.817075 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-26a1cd52-15b3-4a2d-8b5e-c45ba0e8cb09 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:22.818278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-26a1cd52-15b3-4a2d-8b5e-c45ba0e8cb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:22.867778 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:22.894543 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f775218-6b46-4e38-b2c4-38e306ca1dd6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:22.903473 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 257/497] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:22.903473 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a1cd52-15b3-4a2d-8b5e-c45ba0e8cb09 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:22.931284 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67708b-163f-43d9-b100-7c88c72f1633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:22.932129 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67708b-163f-43d9-b100-7c88c72f1633 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:22.932604 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67708b-163f-43d9-b100-7c88c72f1633 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:22.932868 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67708b-163f-43d9-b100-7c88c72f1633 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:22.944121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff67708b-163f-43d9-b100-7c88c72f1633 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:22.949254 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-ff67708b-163f-43d9-b100-7c88c72f1633 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:22.949254 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:22.949254 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:22.949254 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:22.949254 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:22.949254 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:22.949254 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:22.949254 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:22.949254 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:22.949254 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:22.949254 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:22.949254 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:22.949254 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:22.949254 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:22.949254 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:22.949254 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:22.950173 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:22.950173 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:22.950173 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:22.950173 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:22.950173 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:22.950173 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:22.950173 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:22.950173 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:22.950173 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:22.950173 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:22.950173 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:22.950173 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:22.950173 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:22.950173 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:22.950173 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:22.950173 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:22.950972 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:22.950972 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:22.950972 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:22.950972 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:22.950972 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:22.950972 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:22.950972 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:22.950972 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:22.950972 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:22.950972 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:22.950972 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:22.950972 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:22.950972 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 241/498] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 12:23:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:22.952688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67708b-163f-43d9-b100-7c88c72f1633 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:23.002496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:23.019891 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a1cd52-15b3-4a2d-8b5e-c45ba0e8cb09 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:23.057475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67708b-163f-43d9-b100-7c88c72f1633 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:23.058440 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90d43e-1b8f-4b08-acab-730230b92fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:23.059363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90d43e-1b8f-4b08-acab-730230b92fec None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:23.060534 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90d43e-1b8f-4b08-acab-730230b92fec None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:23.060831 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90d43e-1b8f-4b08-acab-730230b92fec None admin] PATH_INFO: `/v3/users/nova` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:23.070208 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90d43e-1b8f-4b08-acab-730230b92fec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:23.072467 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90d43e-1b8f-4b08-acab-730230b92fec None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:23.089363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-1b90d43e-1b8f-4b08-acab-730230b92fec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:23:23.089363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:23.089363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:23.089363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:23.089363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:23.089363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:23.089363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:23.089363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:23.089363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:23.089363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:23.089363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:23.089363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:23.089363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:23:23.089363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:23:23.089363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:23:23.089363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:23:23.090252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:23.090252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:23.090252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:23:23.090252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:23.090252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:23:23.090252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:23.090252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:23.090252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:23.090252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:23.090252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:23.090252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:23.090252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:23.090252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:23.090252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:23.090252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:23.090252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:23.090252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:23.090252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:23.091157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:23.091157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:23.091157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:23:23.091157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:23:23.091157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:23:23.091157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:23:23.091157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:23:23.091157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:23:23.091157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:23:23.091157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:23.091157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 258/499] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Aug 10 12:23:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:23.091750 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90d43e-1b8f-4b08-acab-730230b92fec None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:23.095680 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48041b-b5cf-4770-a768-c838bc418e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:23.096293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48041b-b5cf-4770-a768-c838bc418e06 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:23.096574 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48041b-b5cf-4770-a768-c838bc418e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:23.096847 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48041b-b5cf-4770-a768-c838bc418e06 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:23.097574 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48041b-b5cf-4770-a768-c838bc418e06 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:23.097884 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48041b-b5cf-4770-a768-c838bc418e06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:23.099372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48041b-b5cf-4770-a768-c838bc418e06 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:23.136348 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:23.142019 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 242/500] 10.222.0.49 () {56 vars in 1137 bytes} [Wed Aug 10 12:23:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:23.142019 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48041b-b5cf-4770-a768-c838bc418e06 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:23.205740 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90d43e-1b8f-4b08-acab-730230b92fec None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:23.241637 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5221e18-c98e-423a-862b-4951ac5c402d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:23.242285 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5221e18-c98e-423a-862b-4951ac5c402d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:23.242549 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5221e18-c98e-423a-862b-4951ac5c402d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:23.242826 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5221e18-c98e-423a-862b-4951ac5c402d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:23.248633 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48041b-b5cf-4770-a768-c838bc418e06 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:23.251418 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:23.254141 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5221e18-c98e-423a-862b-4951ac5c402d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:23.261232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-f5221e18-c98e-423a-862b-4951ac5c402d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:23.261232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:23.261232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:23.261232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:23.261232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:23.261232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:23.261232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:23.261232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:23.261232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:23.261232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:23.261232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:23.261232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:23.261232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:23.261232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:23.261232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:23.261232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:23.263693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:23.263693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:23.263693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:23.263693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:23.263693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:23.263693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:23.263693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:23.263693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:23.263693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:23.263693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:23.263693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:23.263693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:23.263693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:23.263693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:23.263693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:23.263693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:23.264494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:23.264494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:23.264494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:23.264494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:23.264494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:23.264494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:23.264494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:23.264494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:23.264494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:23.264494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:23.264494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:23.264494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:23.264494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 259/501] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 12:23:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:23.264494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5221e18-c98e-423a-862b-4951ac5c402d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:23.294301 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d6520-f739-4112-b8d8-2a22fc204e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:23.294908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d6520-f739-4112-b8d8-2a22fc204e9c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:23.295222 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d6520-f739-4112-b8d8-2a22fc204e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:23.295492 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d6520-f739-4112-b8d8-2a22fc204e9c None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:23.296176 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d6520-f739-4112-b8d8-2a22fc204e9c None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:23.296460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d6520-f739-4112-b8d8-2a22fc204e9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:23.297856 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d6520-f739-4112-b8d8-2a22fc204e9c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:23.306167 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 243/502] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 12:23:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:23.306985 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d6520-f739-4112-b8d8-2a22fc204e9c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:23.383347 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:23.387112 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5221e18-c98e-423a-862b-4951ac5c402d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:23.409577 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d6520-f739-4112-b8d8-2a22fc204e9c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:23.427533 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166afab-818a-442b-a034-64b0f68b4b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:23.429102 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166afab-818a-442b-a034-64b0f68b4b41 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:23.429525 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166afab-818a-442b-a034-64b0f68b4b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:23.429954 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166afab-818a-442b-a034-64b0f68b4b41 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:23.430837 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9166afab-818a-442b-a034-64b0f68b4b41 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:23.431311 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9166afab-818a-442b-a034-64b0f68b4b41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:23.433017 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9166afab-818a-442b-a034-64b0f68b4b41 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:23.445674 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1fdd0-7faa-481e-bd37-1b5cec0df2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:23.446287 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1fdd0-7faa-481e-bd37-1b5cec0df2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:23.446619 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1fdd0-7faa-481e-bd37-1b5cec0df2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:23.446894 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1fdd0-7faa-481e-bd37-1b5cec0df2e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:23.478820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 260/503] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Aug 10 12:23:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:23.480135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166afab-818a-442b-a034-64b0f68b4b41 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:23.489801 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:23.539899 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e1fdd0-7faa-481e-bd37-1b5cec0df2e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:23.541499 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e1fdd0-7faa-481e-bd37-1b5cec0df2e3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:23.601487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166afab-818a-442b-a034-64b0f68b4b41 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:23.637908 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb9aceb-7020-42f3-a02c-cd219b11cdb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:23.638516 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9aceb-7020-42f3-a02c-cd219b11cdb0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:23.638794 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9aceb-7020-42f3-a02c-cd219b11cdb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:23.639067 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9aceb-7020-42f3-a02c-cd219b11cdb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:23.639756 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb9aceb-7020-42f3-a02c-cd219b11cdb0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:23.640058 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb9aceb-7020-42f3-a02c-cd219b11cdb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:23.641451 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb9aceb-7020-42f3-a02c-cd219b11cdb0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:23.650490 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 261/504] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 12:23:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:23.651418 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb9aceb-7020-42f3-a02c-cd219b11cdb0 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:23.666785 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 244/505] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:23:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:23:23.666785 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1fdd0-7faa-481e-bd37-1b5cec0df2e3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:23.680079 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:23.749777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:23.764163 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb9aceb-7020-42f3-a02c-cd219b11cdb0 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:23.792341 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1fdd0-7faa-481e-bd37-1b5cec0df2e3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:23.806415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e469074-5b19-4e59-ab37-4a131166ab11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:23.807497 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e469074-5b19-4e59-ab37-4a131166ab11 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:23.807791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e469074-5b19-4e59-ab37-4a131166ab11 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:23.808067 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e469074-5b19-4e59-ab37-4a131166ab11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:23.818346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e469074-5b19-4e59-ab37-4a131166ab11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:23.820503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e469074-5b19-4e59-ab37-4a131166ab11 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:23.828493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd4c13-0b8c-4f2e-980f-3545231a3379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:23.829097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd4c13-0b8c-4f2e-980f-3545231a3379 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:23.829375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd4c13-0b8c-4f2e-980f-3545231a3379 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:23.829641 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd4c13-0b8c-4f2e-980f-3545231a3379 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:23.836399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-0e469074-5b19-4e59-ab37-4a131166ab11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:23.836399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:23.836399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:23.836399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:23.836399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:23.836399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:23.836399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:23.836399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:23.836399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:23.836399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:23.836399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:23.836399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:23.836399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:23.836399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:23.836399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:23.836399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:23.837302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:23.837302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:23.837302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:23.837302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:23.837302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:23.837302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:23.837302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:23.837302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:23.837302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:23.837302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:23.837302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:23.837302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:23.837302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:23.837302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:23.837302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:23.837302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:23.837302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:23.838154 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:23.838154 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:23.838154 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:23.838154 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 262/506] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:23.838742 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ccb9ba-c081-437e-9a47-1546c3b799d4 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:23.838988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ccb9ba-c081-437e-9a47-1546c3b799d4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:23.839280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ccb9ba-c081-437e-9a47-1546c3b799d4 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:23.840520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 263/507] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:23.841209 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ccb9ba-c081-437e-9a47-1546c3b799d4 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:23.907451 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd4c13-0b8c-4f2e-980f-3545231a3379 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:23.909025 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd4c13-0b8c-4f2e-980f-3545231a3379 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:23.921709 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:23.956439 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ccb9ba-c081-437e-9a47-1546c3b799d4 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:23.983214 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:23.998363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05fd405-ccda-49ff-8cca-3fa4803fb7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:23.998993 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05fd405-ccda-49ff-8cca-3fa4803fb7ce None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:23.999306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05fd405-ccda-49ff-8cca-3fa4803fb7ce None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:23.999580 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05fd405-ccda-49ff-8cca-3fa4803fb7ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:24.011136 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05fd405-ccda-49ff-8cca-3fa4803fb7ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:24.012616 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05fd405-ccda-49ff-8cca-3fa4803fb7ce None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:24.025631 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 245/508] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:23:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:23:24.026822 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd4c13-0b8c-4f2e-980f-3545231a3379 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:24.030151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-b05fd405-ccda-49ff-8cca-3fa4803fb7ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:24.030151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:24.030151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:24.030151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:24.030151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:24.030151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:24.030151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:24.030151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:24.030151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:24.030151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:24.030151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:24.030151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:24.030151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:24.030151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:24.030151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:24.030151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:24.031041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:24.031041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:24.031041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:24.031041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:24.031041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:24.031041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:24.031041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:24.031041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:24.031041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:24.031041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:24.031041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:24.031041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:24.031041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:24.031041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:24.031041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:24.031041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:24.031041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:24.031930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:24.031930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:24.031930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:24.031930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 264/509] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:24.032323 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05fd405-ccda-49ff-8cca-3fa4803fb7ce None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:24.145617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05fd405-ccda-49ff-8cca-3fa4803fb7ce None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:24.149625 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd4c13-0b8c-4f2e-980f-3545231a3379 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:24.169527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:24.181368 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc2f97-5eb8-4f6b-a866-19bbf7f22929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:24.182234 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc2f97-5eb8-4f6b-a866-19bbf7f22929 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:24.182234 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc2f97-5eb8-4f6b-a866-19bbf7f22929 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:24.187001 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc2f97-5eb8-4f6b-a866-19bbf7f22929 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:24.187001 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc2f97-5eb8-4f6b-a866-19bbf7f22929 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:24.187001 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc2f97-5eb8-4f6b-a866-19bbf7f22929 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:24.187001 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc2f97-5eb8-4f6b-a866-19bbf7f22929 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:24.190199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c67b26-8417-4a6f-9015-72e6417c27fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:24.190971 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c67b26-8417-4a6f-9015-72e6417c27fd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:24.191216 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c67b26-8417-4a6f-9015-72e6417c27fd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:24.191437 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c67b26-8417-4a6f-9015-72e6417c27fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:24.197029 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 265/510] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:24.198332 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa13cde-987a-4152-8b9a-89d6af8941a1 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:24.198843 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa13cde-987a-4152-8b9a-89d6af8941a1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:24.198947 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa13cde-987a-4152-8b9a-89d6af8941a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:24.202601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c67b26-8417-4a6f-9015-72e6417c27fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:24.204093 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c67b26-8417-4a6f-9015-72e6417c27fd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:24.216953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-faa13cde-987a-4152-8b9a-89d6af8941a1 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:24.218159 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-faa13cde-987a-4152-8b9a-89d6af8941a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:24.218462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-83c67b26-8417-4a6f-9015-72e6417c27fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:24.218462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:24.218462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:24.218462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:24.218462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:24.218462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:24.218462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:24.218462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:24.218462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:24.218462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:24.218462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:24.218462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:24.218462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:24.218462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:24.218462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:24.218462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:24.219544 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:24.219544 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:24.219544 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:24.219544 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:24.219544 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:24.219544 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:24.219544 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:24.219544 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:24.219544 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:24.219544 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:24.219544 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:24.219544 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:24.219544 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:24.219544 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:24.219544 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:24.219544 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:24.219544 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:24.220390 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:24.220390 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:24.220390 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:24.220390 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 246/511] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:24.220390 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c67b26-8417-4a6f-9015-72e6417c27fd None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:24.238764 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:24.293912 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 266/512] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:24.294764 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa13cde-987a-4152-8b9a-89d6af8941a1 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:24.328570 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c67b26-8417-4a6f-9015-72e6417c27fd None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:24.362829 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1bb30e-5faf-40e4-a6d5-c8ae5766d8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:24.363490 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1bb30e-5faf-40e4-a6d5-c8ae5766d8df None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:24.363774 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1bb30e-5faf-40e4-a6d5-c8ae5766d8df None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:24.364049 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1bb30e-5faf-40e4-a6d5-c8ae5766d8df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:24.375517 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1bb30e-5faf-40e4-a6d5-c8ae5766d8df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:24.376970 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1bb30e-5faf-40e4-a6d5-c8ae5766d8df None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:24.392105 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-3b1bb30e-5faf-40e4-a6d5-c8ae5766d8df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:24.392105 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:24.392105 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:24.392105 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:24.392105 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:24.392105 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:24.392105 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:24.392105 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:24.392105 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:24.392105 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:24.392105 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:24.392105 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:24.392105 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:24.392105 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:24.392105 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:24.392105 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:24.393020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:24.393020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:24.393020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:24.393020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:24.393020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:24.393020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:24.393020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:24.393020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:24.393020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:24.393020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:24.393020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:24.393020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:24.393020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:24.393020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:24.393020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:24.393020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:24.393020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:24.393850 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:24.393850 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:24.393850 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:24.393850 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 247/513] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:24.394083 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1bb30e-5faf-40e4-a6d5-c8ae5766d8df None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:24.397374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa13cde-987a-4152-8b9a-89d6af8941a1 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:24.411216 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:24.431556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc86740-1916-4f9a-b322-f228a34337d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:24.432162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc86740-1916-4f9a-b322-f228a34337d9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:24.432443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc86740-1916-4f9a-b322-f228a34337d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:24.432716 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc86740-1916-4f9a-b322-f228a34337d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:24.433356 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc86740-1916-4f9a-b322-f228a34337d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:24.433664 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc86740-1916-4f9a-b322-f228a34337d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:24.435095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc86740-1916-4f9a-b322-f228a34337d9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:24.447359 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 267/514] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:24.452433 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc86740-1916-4f9a-b322-f228a34337d9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:24.463304 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:24.493585 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1bb30e-5faf-40e4-a6d5-c8ae5766d8df None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:24.526939 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823664fa-5c43-4f4a-87bb-4268c3aaf323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:24.527603 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823664fa-5c43-4f4a-87bb-4268c3aaf323 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:24.527884 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823664fa-5c43-4f4a-87bb-4268c3aaf323 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:24.528162 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823664fa-5c43-4f4a-87bb-4268c3aaf323 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:24.535666 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823664fa-5c43-4f4a-87bb-4268c3aaf323 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:24.537115 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823664fa-5c43-4f4a-87bb-4268c3aaf323 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:24.549932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-823664fa-5c43-4f4a-87bb-4268c3aaf323 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 12:23:24.549932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:24.549932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:24.549932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:24.549932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:24.549932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:24.549932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:24.549932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:24.549932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:24.549932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:24.549932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:24.549932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:24.549932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:23:24.549932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:23:24.549932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:23:24.549932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:23:24.550825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:24.550825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:24.550825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:23:24.550825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:24.550825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:23:24.550825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:24.550825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:24.550825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:24.550825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:24.550825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:24.550825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:24.550825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:24.550825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:24.550825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:24.550825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:24.550825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:24.550825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:24.550825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:24.551752 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:24.551752 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:24.551752 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:23:24.551752 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:23:24.551752 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:23:24.551752 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:23:24.551752 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:23:24.551752 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:23:24.551752 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 12:23:24.551752 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:24.551752 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 248/515] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 12:23:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:24.552304 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823664fa-5c43-4f4a-87bb-4268c3aaf323 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:24.558493 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc86740-1916-4f9a-b322-f228a34337d9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:24.596655 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ba4ff-2682-47cd-8773-9e52e02b2801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:24.597270 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ba4ff-2682-47cd-8773-9e52e02b2801 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:24.597553 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ba4ff-2682-47cd-8773-9e52e02b2801 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:24.597818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ba4ff-2682-47cd-8773-9e52e02b2801 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:24.598450 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ba4ff-2682-47cd-8773-9e52e02b2801 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:24.598750 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ba4ff-2682-47cd-8773-9e52e02b2801 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:24.600171 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ba4ff-2682-47cd-8773-9e52e02b2801 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:24.612236 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 268/516] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:24.613047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ba4ff-2682-47cd-8773-9e52e02b2801 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:24.648502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:24.659302 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823664fa-5c43-4f4a-87bb-4268c3aaf323 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:24.687672 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:24.693508 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134c6ee-afd0-4f64-9f5f-73f5ead6db9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:24.694736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134c6ee-afd0-4f64-9f5f-73f5ead6db9c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:24.695018 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134c6ee-afd0-4f64-9f5f-73f5ead6db9c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:24.695330 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134c6ee-afd0-4f64-9f5f-73f5ead6db9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:24.705625 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c134c6ee-afd0-4f64-9f5f-73f5ead6db9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:24.707087 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c134c6ee-afd0-4f64-9f5f-73f5ead6db9c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:24.714228 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ba4ff-2682-47cd-8773-9e52e02b2801 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:24.721883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-c134c6ee-afd0-4f64-9f5f-73f5ead6db9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:24.721883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:24.721883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:24.721883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:24.721883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:24.721883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:24.721883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:24.721883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:24.721883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:24.721883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:24.721883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:24.721883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:24.721883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:24.721883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:24.721883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:24.721883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:24.722800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:24.722800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:24.722800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:24.722800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:24.722800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:24.722800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:24.722800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:24.722800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:24.722800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:24.722800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:24.722800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:24.722800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:24.722800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:24.722800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:24.722800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:24.722800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:24.722800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:24.723689 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:24.723689 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:24.723689 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:24.723689 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 249/517] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:24.723912 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134c6ee-afd0-4f64-9f5f-73f5ead6db9c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:24.752814 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb82d172-639a-4584-80d7-70ed352300ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:24.753425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82d172-639a-4584-80d7-70ed352300ed None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:24.753707 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82d172-639a-4584-80d7-70ed352300ed None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:24.753975 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82d172-639a-4584-80d7-70ed352300ed None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:24.754614 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb82d172-639a-4584-80d7-70ed352300ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:24.754916 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb82d172-639a-4584-80d7-70ed352300ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:24.756866 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb82d172-639a-4584-80d7-70ed352300ed None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:24.768080 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 269/518] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:24.768967 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb82d172-639a-4584-80d7-70ed352300ed None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:24.826208 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134c6ee-afd0-4f64-9f5f-73f5ead6db9c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:24.860177 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d55d415-edd0-450f-a75c-3e8a12a33db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:24.860801 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55d415-edd0-450f-a75c-3e8a12a33db4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:24.861086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55d415-edd0-450f-a75c-3e8a12a33db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:24.861190 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:24.861358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55d415-edd0-450f-a75c-3e8a12a33db4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:24.868398 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb82d172-639a-4584-80d7-70ed352300ed None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:24.869338 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d55d415-edd0-450f-a75c-3e8a12a33db4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:24.873889 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d55d415-edd0-450f-a75c-3e8a12a33db4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:24.881954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-7d55d415-edd0-450f-a75c-3e8a12a33db4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 12:23:24.881954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:24.881954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:24.881954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:24.881954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:24.881954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:24.881954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:24.881954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:24.881954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:24.881954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:24.881954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:24.881954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:24.881954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:23:24.881954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:23:24.881954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:23:24.881954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:23:24.883030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:24.883030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:24.883030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:23:24.883030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:24.883030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:23:24.883030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:24.883030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:24.883030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:24.883030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:24.883030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:24.883030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:24.883030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:24.883030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:24.883030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:24.883030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:24.883030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:24.883030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:24.883030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:24.884187 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:24.884187 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:24.884187 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:23:24.884187 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:23:24.884187 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:23:24.884187 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:23:24.884187 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:23:24.884187 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:23:24.884187 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 12:23:24.884187 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:24.884187 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 250/519] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:23:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:24.884187 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d55d415-edd0-450f-a75c-3e8a12a33db4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:24.906777 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cb986c-54b8-4db5-b9f2-51a11fd4f3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:24.907357 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb986c-54b8-4db5-b9f2-51a11fd4f3ac None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:24.907640 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb986c-54b8-4db5-b9f2-51a11fd4f3ac None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:24.907913 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb986c-54b8-4db5-b9f2-51a11fd4f3ac None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:24.908641 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cb986c-54b8-4db5-b9f2-51a11fd4f3ac None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:24.908949 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cb986c-54b8-4db5-b9f2-51a11fd4f3ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:24.910369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cb986c-54b8-4db5-b9f2-51a11fd4f3ac None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:24.926527 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:24.927553 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 270/520] 10.222.0.49 () {56 vars in 1139 bytes} [Wed Aug 10 12:23:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:24.929112 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cb986c-54b8-4db5-b9f2-51a11fd4f3ac None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:24.995191 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d55d415-edd0-450f-a75c-3e8a12a33db4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:25.034489 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ce05a-7228-4f79-9318-b1fe032e525e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:25.035096 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cb986c-54b8-4db5-b9f2-51a11fd4f3ac None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:25.035517 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ce05a-7228-4f79-9318-b1fe032e525e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:25.035976 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ce05a-7228-4f79-9318-b1fe032e525e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:25.036428 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ce05a-7228-4f79-9318-b1fe032e525e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:25.044232 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ce05a-7228-4f79-9318-b1fe032e525e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:25.045665 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ce05a-7228-4f79-9318-b1fe032e525e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:25.057969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-1d6ce05a-7228-4f79-9318-b1fe032e525e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:25.057969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:25.057969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:25.057969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:25.057969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:25.057969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:25.057969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:25.057969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:25.057969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:25.057969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:25.057969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:25.057969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:25.057969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:23:25.057969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:23:25.057969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:23:25.059065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:23:25.059065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:25.059065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:25.059065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:25.059065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:25.059065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:25.059065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:25.059065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:25.059065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:25.059065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:25.059065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:25.059065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:25.059065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:25.059065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:25.059065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:25.059065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:25.059065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:25.065106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:23:25.065106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:23:25.065106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:23:25.065106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:23:25.065106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:23:25.065106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:23:25.065106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:25.065106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:25.065106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 251/521] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:25.065106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ce05a-7228-4f79-9318-b1fe032e525e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:25.070743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecab3a-d7c8-45d8-9c2c-6025b550ab2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:25.071733 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecab3a-d7c8-45d8-9c2c-6025b550ab2a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:25.071733 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecab3a-d7c8-45d8-9c2c-6025b550ab2a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:25.071940 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecab3a-d7c8-45d8-9c2c-6025b550ab2a None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:25.072552 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecab3a-d7c8-45d8-9c2c-6025b550ab2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:25.072854 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecab3a-d7c8-45d8-9c2c-6025b550ab2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:25.074277 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecab3a-d7c8-45d8-9c2c-6025b550ab2a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:25.085818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 271/522] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:25.086605 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecab3a-d7c8-45d8-9c2c-6025b550ab2a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:25.101662 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:25.166200 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ce05a-7228-4f79-9318-b1fe032e525e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:25.167678 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:25.185708 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecab3a-d7c8-45d8-9c2c-6025b550ab2a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:25.199684 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc27af-1d21-4ea8-9b69-7c0bec94840e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:25.200628 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc27af-1d21-4ea8-9b69-7c0bec94840e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:25.201069 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc27af-1d21-4ea8-9b69-7c0bec94840e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:25.201647 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc27af-1d21-4ea8-9b69-7c0bec94840e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:25.208907 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc27af-1d21-4ea8-9b69-7c0bec94840e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:25.210897 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc27af-1d21-4ea8-9b69-7c0bec94840e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:25.218726 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9761c922-a6b5-4f4a-8b12-dabe0cca5dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:25.219358 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9761c922-a6b5-4f4a-8b12-dabe0cca5dde None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:25.219636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9761c922-a6b5-4f4a-8b12-dabe0cca5dde None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:25.219909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9761c922-a6b5-4f4a-8b12-dabe0cca5dde None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:25.220623 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9761c922-a6b5-4f4a-8b12-dabe0cca5dde None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:25.220927 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9761c922-a6b5-4f4a-8b12-dabe0cca5dde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:25.221853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-c9fc27af-1d21-4ea8-9b69-7c0bec94840e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:25.221853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:25.221853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:25.221853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:25.221853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:25.221853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:25.221853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:25.221853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:25.221853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:25.221853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:25.221853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:25.221853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:25.221853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:23:25.221853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:23:25.221853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:23:25.222824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:23:25.222824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:25.222824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:25.222824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:25.222824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:25.222824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:25.222824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:25.222824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:25.222824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:25.222824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:25.222824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:25.222824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:25.222824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:25.222824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:25.222824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:25.222824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:25.222824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:25.223839 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:23:25.223839 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:23:25.223839 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:23:25.223839 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:23:25.223839 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:23:25.223839 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:23:25.223839 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:25.223839 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:25.224495 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9761c922-a6b5-4f4a-8b12-dabe0cca5dde None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:25.224566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 252/523] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:25.224566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc27af-1d21-4ea8-9b69-7c0bec94840e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:25.240778 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 272/524] 10.222.0.49 () {56 vars in 1137 bytes} [Wed Aug 10 12:23:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:25.241580 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9761c922-a6b5-4f4a-8b12-dabe0cca5dde None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:25.309855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:25.329765 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc27af-1d21-4ea8-9b69-7c0bec94840e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:25.339813 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9761c922-a6b5-4f4a-8b12-dabe0cca5dde None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:25.363707 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941c538-2c6f-48fe-9b11-1bc8d23c32aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:25.364313 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941c538-2c6f-48fe-9b11-1bc8d23c32aa None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:25.364595 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941c538-2c6f-48fe-9b11-1bc8d23c32aa None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:25.364870 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941c538-2c6f-48fe-9b11-1bc8d23c32aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:25.373828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b745383-829a-4f70-adc7-daec8822825c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:25.374425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b745383-829a-4f70-adc7-daec8822825c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:25.374702 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b745383-829a-4f70-adc7-daec8822825c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:25.374972 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b745383-829a-4f70-adc7-daec8822825c None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:25.375686 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b745383-829a-4f70-adc7-daec8822825c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:25.375989 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b745383-829a-4f70-adc7-daec8822825c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:25.377397 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b745383-829a-4f70-adc7-daec8822825c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:25.393816 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:25.395016 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 273/525] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:23:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:25.395704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b745383-829a-4f70-adc7-daec8822825c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:25.432251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0941c538-2c6f-48fe-9b11-1bc8d23c32aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:25.433658 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0941c538-2c6f-48fe-9b11-1bc8d23c32aa None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:25.496955 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b745383-829a-4f70-adc7-daec8822825c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:25.526249 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:25.531739 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 253/526] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:23:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:23:25.532189 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54306e07-b8b7-4cd3-a412-8b22ed9ccd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:25.532619 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941c538-2c6f-48fe-9b11-1bc8d23c32aa None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:25.532731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54306e07-b8b7-4cd3-a412-8b22ed9ccd07 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:25.532977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54306e07-b8b7-4cd3-a412-8b22ed9ccd07 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:25.533208 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54306e07-b8b7-4cd3-a412-8b22ed9ccd07 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:25.533850 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54306e07-b8b7-4cd3-a412-8b22ed9ccd07 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:25.535315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54306e07-b8b7-4cd3-a412-8b22ed9ccd07 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:25.594952 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:25.620253 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 274/527] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 12:23:25 2022] POST /identity/v3/users => generated 247 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:23:25.621341 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54306e07-b8b7-4cd3-a412-8b22ed9ccd07 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:25.635225 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941c538-2c6f-48fe-9b11-1bc8d23c32aa None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:25.674213 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665eeeb-2613-4704-a2eb-8965700460e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:25.674836 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665eeeb-2613-4704-a2eb-8965700460e2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:25.675115 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665eeeb-2613-4704-a2eb-8965700460e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:25.675419 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665eeeb-2613-4704-a2eb-8965700460e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:25.676130 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7665eeeb-2613-4704-a2eb-8965700460e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:25.676435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7665eeeb-2613-4704-a2eb-8965700460e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:25.677943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7665eeeb-2613-4704-a2eb-8965700460e2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:25.698033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 254/528] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:23:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:25.698823 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665eeeb-2613-4704-a2eb-8965700460e2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:25.743886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54306e07-b8b7-4cd3-a412-8b22ed9ccd07 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:25.783033 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a5d93-8685-4d20-b8c2-0400c025e10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:25.783682 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a5d93-8685-4d20-b8c2-0400c025e10e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:25.783971 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a5d93-8685-4d20-b8c2-0400c025e10e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:25.784240 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a5d93-8685-4d20-b8c2-0400c025e10e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:25.824319 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665eeeb-2613-4704-a2eb-8965700460e2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:25.846285 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:25.849763 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:25.860265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a5d93-8685-4d20-b8c2-0400c025e10e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:25.861748 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a5d93-8685-4d20-b8c2-0400c025e10e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:25.862328 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7eead2-56b7-491e-a163-39ca1033dccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:25.862933 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7eead2-56b7-491e-a163-39ca1033dccf None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:25.874202 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7eead2-56b7-491e-a163-39ca1033dccf None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:25.874202 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7eead2-56b7-491e-a163-39ca1033dccf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:25.874202 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7eead2-56b7-491e-a163-39ca1033dccf None admin] RBAC: Adding query filter params (role.id=a9bbb011afca4176a3def9356cb5f165, scope.project.id=8e201e5f281a4977be503e22daf149d7, user.id=ab5a0d9c39d84cc7a91f5a752dcf2b70) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:25.874202 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7eead2-56b7-491e-a163-39ca1033dccf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:25.874202 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7eead2-56b7-491e-a163-39ca1033dccf None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:25.874546 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 255/529] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:23:25 2022] GET /identity/v3/role_assignments?user.id=ab5a0d9c39d84cc7a91f5a752dcf2b70&scope.project.id=8e201e5f281a4977be503e22daf149d7&role.id=a9bbb011afca4176a3def9356cb5f165 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:25.874546 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7eead2-56b7-491e-a163-39ca1033dccf None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:26.002109 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 275/530] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:23:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:23:26.002109 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a5d93-8685-4d20-b8c2-0400c025e10e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:26.019664 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7eead2-56b7-491e-a163-39ca1033dccf None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:26.064542 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f26643-be11-41ea-95f8-675a6b51593a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:26.065663 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f26643-be11-41ea-95f8-675a6b51593a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:26.066149 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f26643-be11-41ea-95f8-675a6b51593a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:26.066612 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f26643-be11-41ea-95f8-675a6b51593a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:26.078593 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f26643-be11-41ea-95f8-675a6b51593a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:26.080482 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f26643-be11-41ea-95f8-675a6b51593a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:26.097440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a1f26643-be11-41ea-95f8-675a6b51593a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:26.097440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:26.097440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:26.097440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:26.097440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:26.097440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:26.097440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:26.097440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:26.097440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:26.097440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:26.097440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:26.097440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:26.097440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:26.097440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:26.097440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:26.097440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:26.098373 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:26.098373 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:26.098373 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:26.098373 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:26.098373 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:26.098373 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:26.098373 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:26.098373 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:26.098373 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:26.098373 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:26.098373 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:26.098373 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:26.098373 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:26.098373 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:26.098373 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:26.098373 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:26.098373 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:26.099236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:26.099236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:26.099236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:26.102234 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 256/531] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:26.103408 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f26643-be11-41ea-95f8-675a6b51593a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:26.121717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:26.138059 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a5d93-8685-4d20-b8c2-0400c025e10e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:26.158112 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:26.175311 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c574cc6-8270-4615-96db-d4dda1085b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:26.175934 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c574cc6-8270-4615-96db-d4dda1085b4b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:26.176225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c574cc6-8270-4615-96db-d4dda1085b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:26.176501 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c574cc6-8270-4615-96db-d4dda1085b4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:26.177290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c574cc6-8270-4615-96db-d4dda1085b4b None admin] RBAC: Adding query filter params (role.id=a9bbb011afca4176a3def9356cb5f165, scope.project.id=8e201e5f281a4977be503e22daf149d7, user.id=eb3f6deffb54467f8b923956ed2a6acf) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:26.177534 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c574cc6-8270-4615-96db-d4dda1085b4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:26.179588 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c574cc6-8270-4615-96db-d4dda1085b4b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:26.185691 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 276/532] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:23:25 2022] GET /identity/v3/role_assignments?user.id=eb3f6deffb54467f8b923956ed2a6acf&scope.project.id=8e201e5f281a4977be503e22daf149d7&role.id=a9bbb011afca4176a3def9356cb5f165 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:26.186601 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c574cc6-8270-4615-96db-d4dda1085b4b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:26.242789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f26643-be11-41ea-95f8-675a6b51593a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:26.294640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdaf1d0-0843-4e91-9e2c-9dbe56ea79e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:26.295496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdaf1d0-0843-4e91-9e2c-9dbe56ea79e8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:26.295848 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdaf1d0-0843-4e91-9e2c-9dbe56ea79e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:26.296196 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdaf1d0-0843-4e91-9e2c-9dbe56ea79e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:26.308428 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdaf1d0-0843-4e91-9e2c-9dbe56ea79e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:26.308603 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c574cc6-8270-4615-96db-d4dda1085b4b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:26.310118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdaf1d0-0843-4e91-9e2c-9dbe56ea79e8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:26.328498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-dcdaf1d0-0843-4e91-9e2c-9dbe56ea79e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:26.328498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:26.328498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:26.328498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:26.328498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:26.328498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:26.328498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:26.328498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:26.328498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:26.328498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:26.328498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:26.328498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:26.328498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:26.328498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:26.328498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:26.328498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:26.329463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:26.329463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:26.329463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:26.329463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:26.329463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:26.329463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:26.329463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:26.329463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:26.329463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:26.329463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:26.329463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:26.329463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:26.329463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:26.329463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:26.329463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:26.329463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:26.329463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:26.330297 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:26.330297 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:26.330297 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:26.330297 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 257/533] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:26.340411 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdaf1d0-0843-4e91-9e2c-9dbe56ea79e8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:26.362955 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f05cff-f23c-4031-a57d-3f5c17ba3ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:26.364218 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f05cff-f23c-4031-a57d-3f5c17ba3ceb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:26.364503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f05cff-f23c-4031-a57d-3f5c17ba3ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:26.364781 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f05cff-f23c-4031-a57d-3f5c17ba3ceb None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:26.365457 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f05cff-f23c-4031-a57d-3f5c17ba3ceb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:26.365882 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f05cff-f23c-4031-a57d-3f5c17ba3ceb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:26.368037 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f05cff-f23c-4031-a57d-3f5c17ba3ceb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:26.383706 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 277/534] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:26.388596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f05cff-f23c-4031-a57d-3f5c17ba3ceb None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:26.413353 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:26.521582 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:26.538029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdaf1d0-0843-4e91-9e2c-9dbe56ea79e8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:26.546128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f05cff-f23c-4031-a57d-3f5c17ba3ceb None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:26.587314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227dfd1e-6df9-4d7b-bcf8-bbc75a82418e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:26.587929 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227dfd1e-6df9-4d7b-bcf8-bbc75a82418e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:26.588209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227dfd1e-6df9-4d7b-bcf8-bbc75a82418e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:26.588483 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227dfd1e-6df9-4d7b-bcf8-bbc75a82418e None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:26.589121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227dfd1e-6df9-4d7b-bcf8-bbc75a82418e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:26.589424 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227dfd1e-6df9-4d7b-bcf8-bbc75a82418e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:26.590801 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227dfd1e-6df9-4d7b-bcf8-bbc75a82418e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:26.603790 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 258/535] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:26.611482 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227dfd1e-6df9-4d7b-bcf8-bbc75a82418e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:26.641285 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af5789f-2155-4d7c-8e48-79aefc6ad3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:26.641950 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5789f-2155-4d7c-8e48-79aefc6ad3db None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:26.642210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5789f-2155-4d7c-8e48-79aefc6ad3db None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:26.642487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5789f-2155-4d7c-8e48-79aefc6ad3db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:26.654169 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af5789f-2155-4d7c-8e48-79aefc6ad3db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:26.655685 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af5789f-2155-4d7c-8e48-79aefc6ad3db None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:26.685461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-7af5789f-2155-4d7c-8e48-79aefc6ad3db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:26.685461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:26.685461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:26.685461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:26.685461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:26.685461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:26.685461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:26.685461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:26.685461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:26.685461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:26.685461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:26.685461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:26.685461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:23:26.685461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:23:26.685461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:23:26.686345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:23:26.686345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:26.686345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:26.686345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:26.686345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:26.686345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:26.686345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:26.686345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:26.686345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:26.686345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:26.686345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:26.686345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:26.686345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:26.686345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:26.686345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:26.686345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:26.686345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:26.687282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:23:26.687282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:23:26.687282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:23:26.687282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:23:26.687282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:23:26.687282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:23:26.687282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:26.687282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:26.687282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 278/536] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:23:26.692255 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af5789f-2155-4d7c-8e48-79aefc6ad3db None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:26.720403 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227dfd1e-6df9-4d7b-bcf8-bbc75a82418e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:26.756212 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a84db-83f0-4eb0-8c09-e2501967d82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:26.756853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a84db-83f0-4eb0-8c09-e2501967d82f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:26.757135 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a84db-83f0-4eb0-8c09-e2501967d82f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:26.757477 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a84db-83f0-4eb0-8c09-e2501967d82f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:26.764714 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a84db-83f0-4eb0-8c09-e2501967d82f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:26.766170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a84db-83f0-4eb0-8c09-e2501967d82f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:26.778392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-905a84db-83f0-4eb0-8c09-e2501967d82f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:26.778392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:26.778392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:26.778392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:26.778392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:26.778392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:26.778392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:26.778392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:26.778392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:26.778392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:26.778392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:26.778392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:26.778392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:23:26.778392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:23:26.778392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:23:26.779277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:23:26.779277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:26.779277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:26.779277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:26.779277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:26.779277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:26.779277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:26.779277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:26.779277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:26.779277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:26.779277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:26.779277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:26.779277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:26.779277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:26.779277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:26.779277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:26.779277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:26.780120 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:23:26.780120 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:23:26.780120 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:23:26.780120 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:23:26.780120 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:23:26.780120 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:23:26.780120 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:26.780120 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:26.780120 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 259/537] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:26.788386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a84db-83f0-4eb0-8c09-e2501967d82f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:26.812257 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:26.855544 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:26.910854 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a84db-83f0-4eb0-8c09-e2501967d82f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:26.948647 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f228a73-39bc-4dfb-853f-a8194c17f5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:26.949609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f228a73-39bc-4dfb-853f-a8194c17f5ea None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:26.949895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f228a73-39bc-4dfb-853f-a8194c17f5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:26.950172 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f228a73-39bc-4dfb-853f-a8194c17f5ea None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:26.950864 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f228a73-39bc-4dfb-853f-a8194c17f5ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:26.951194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f228a73-39bc-4dfb-853f-a8194c17f5ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:26.952995 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f228a73-39bc-4dfb-853f-a8194c17f5ea None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:26.957122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af5789f-2155-4d7c-8e48-79aefc6ad3db None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:26.966229 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 260/538] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:23:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:26.972135 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f228a73-39bc-4dfb-853f-a8194c17f5ea None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:27.034085 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65807780-a88d-457d-9e62-2c785257d26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:27.034710 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65807780-a88d-457d-9e62-2c785257d26a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:27.034995 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65807780-a88d-457d-9e62-2c785257d26a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:27.035308 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65807780-a88d-457d-9e62-2c785257d26a None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:27.036040 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65807780-a88d-457d-9e62-2c785257d26a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:27.036424 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65807780-a88d-457d-9e62-2c785257d26a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:27.039020 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65807780-a88d-457d-9e62-2c785257d26a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:27.063074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:27.065489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 279/539] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:23:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:23:27.072782 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65807780-a88d-457d-9e62-2c785257d26a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:27.127893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f228a73-39bc-4dfb-853f-a8194c17f5ea None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:27.188225 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889ca39-5453-470d-a996-257ab9b167e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:27.188844 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889ca39-5453-470d-a996-257ab9b167e1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:27.189127 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889ca39-5453-470d-a996-257ab9b167e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:27.189402 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889ca39-5453-470d-a996-257ab9b167e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:27.190138 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8889ca39-5453-470d-a996-257ab9b167e1 None admin] RBAC: Adding query filter params (role.id=a9bbb011afca4176a3def9356cb5f165, scope.project.id=8e201e5f281a4977be503e22daf149d7, user.id=35dd27fd356241b885706375d880222f) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:27.190453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8889ca39-5453-470d-a996-257ab9b167e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:27.191971 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8889ca39-5453-470d-a996-257ab9b167e1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:27.199132 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 261/540] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:23:26 2022] GET /identity/v3/role_assignments?user.id=35dd27fd356241b885706375d880222f&scope.project.id=8e201e5f281a4977be503e22daf149d7&role.id=a9bbb011afca4176a3def9356cb5f165 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:23:27.271553 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65807780-a88d-457d-9e62-2c785257d26a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:27.324801 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:27.334914 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a9c76-d5ec-42ff-abb0-9afb8b86a12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:27.335580 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a9c76-d5ec-42ff-abb0-9afb8b86a12f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:27.335864 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a9c76-d5ec-42ff-abb0-9afb8b86a12f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:27.336141 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a9c76-d5ec-42ff-abb0-9afb8b86a12f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:27.336887 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a9c76-d5ec-42ff-abb0-9afb8b86a12f None admin] RBAC: Adding query filter params (role.id=a9bbb011afca4176a3def9356cb5f165, scope.project.id=8e201e5f281a4977be503e22daf149d7, user.id=5db814952f9f466abf6148b7a0db2173) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:27.341230 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a9c76-d5ec-42ff-abb0-9afb8b86a12f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:27.341230 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a9c76-d5ec-42ff-abb0-9afb8b86a12f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:27.347677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 280/541] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:23:27 2022] GET /identity/v3/role_assignments?user.id=5db814952f9f466abf6148b7a0db2173&scope.project.id=8e201e5f281a4977be503e22daf149d7&role.id=a9bbb011afca4176a3def9356cb5f165 => generated 259 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:23:28.103790 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f62279-e23e-4dee-b2ad-165789a945f9 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:28.104304 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f62279-e23e-4dee-b2ad-165789a945f9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:28.104411 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f62279-e23e-4dee-b2ad-165789a945f9 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:28.106028 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 262/542] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:28.111517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f0235-65b6-485b-aa7d-d9b81f5400c4 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:28.111928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f0235-65b6-485b-aa7d-d9b81f5400c4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:28.112118 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f0235-65b6-485b-aa7d-d9b81f5400c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:28.131615 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ab6f0235-65b6-485b-aa7d-d9b81f5400c4 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:28.132825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ab6f0235-65b6-485b-aa7d-d9b81f5400c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:28.207291 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 281/543] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:28.227236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f62279-e23e-4dee-b2ad-165789a945f9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:28.250068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40465b7-890c-4ec9-b32f-db1e5f822605 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:28.250418 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40465b7-890c-4ec9-b32f-db1e5f822605 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:28.250702 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40465b7-890c-4ec9-b32f-db1e5f822605 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:28.252205 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 282/544] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:23:28.263114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38433f-b38f-436a-9f02-fb77e7b5e105 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:28.263421 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38433f-b38f-436a-9f02-fb77e7b5e105 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:28.263690 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38433f-b38f-436a-9f02-fb77e7b5e105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:28.289456 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0c38433f-b38f-436a-9f02-fb77e7b5e105 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:28.290680 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0c38433f-b38f-436a-9f02-fb77e7b5e105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:28.409013 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f62279-e23e-4dee-b2ad-165789a945f9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:28.441999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 283/545] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:28.454785 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7d777-e8ca-4564-8a40-61918c2dc6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:28.455537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7d777-e8ca-4564-8a40-61918c2dc6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:28.455827 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7d777-e8ca-4564-8a40-61918c2dc6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:28.456103 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7d777-e8ca-4564-8a40-61918c2dc6b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:28.461363 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:28.467644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7d777-e8ca-4564-8a40-61918c2dc6b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:28.473474 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38433f-b38f-436a-9f02-fb77e7b5e105 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:28.479872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-cea7d777-e8ca-4564-8a40-61918c2dc6b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:28.479872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:28.479872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:28.479872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:28.479872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:28.479872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:28.479872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:28.479872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:28.479872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:28.479872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:28.479872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:28.479872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:28.479872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:28.479872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:28.479872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:28.479872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:28.480755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:28.480755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:28.480755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:28.480755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:28.480755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:28.480755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:28.480755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:28.480755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:28.480755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:28.480755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:28.480755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:28.480755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:28.480755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:28.480755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:28.480755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:28.480755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:28.481551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:28.481551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:28.481551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:28.481551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:28.481551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:28.481551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:28.481551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:28.481551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:28.481551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:28.481551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:28.481551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:28.481551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:28.483789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 263/546] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 12:23:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:28.489570 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7d777-e8ca-4564-8a40-61918c2dc6b3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:28.535662 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:28.586806 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38433f-b38f-436a-9f02-fb77e7b5e105 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:28.624591 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292969c-48b4-4279-a9af-0a96ed9ea1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:28.625239 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292969c-48b4-4279-a9af-0a96ed9ea1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:28.625519 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292969c-48b4-4279-a9af-0a96ed9ea1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:28.625795 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292969c-48b4-4279-a9af-0a96ed9ea1d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:28.643212 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f292969c-48b4-4279-a9af-0a96ed9ea1d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:28.644195 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-f292969c-48b4-4279-a9af-0a96ed9ea1d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:28.644195 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:28.644195 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:28.644195 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:28.644195 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:28.644195 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:28.644195 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:28.644195 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:28.644195 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:28.644195 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:28.644195 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:28.644195 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:28.644195 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:28.644195 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:28.644195 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:28.644195 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:28.645070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:28.645070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:28.645070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:28.645070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:28.645070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:28.645070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:28.645070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:28.645070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:28.645070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:28.645070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:28.645070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:28.645070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:28.645070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:28.645070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:28.645070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:28.645070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:28.645854 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:28.645854 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:28.645854 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:28.645854 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:28.645854 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:28.645854 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:28.645854 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:28.645854 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:28.645854 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:28.645854 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:28.645854 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:28.645854 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:28.645854 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 284/547] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 12:23:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:23:28.651719 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292969c-48b4-4279-a9af-0a96ed9ea1d4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:28.712222 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7d777-e8ca-4564-8a40-61918c2dc6b3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:28.764786 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0147c-73bb-45bc-b241-88c44160b683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:28.765391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0147c-73bb-45bc-b241-88c44160b683 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:28.765668 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0147c-73bb-45bc-b241-88c44160b683 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:28.765941 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0147c-73bb-45bc-b241-88c44160b683 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:28.766596 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0147c-73bb-45bc-b241-88c44160b683 None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:28.767113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0147c-73bb-45bc-b241-88c44160b683 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:28.768321 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0147c-73bb-45bc-b241-88c44160b683 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:28.787119 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 264/548] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 12:23:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:23:28.796136 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0147c-73bb-45bc-b241-88c44160b683 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:28.813358 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292969c-48b4-4279-a9af-0a96ed9ea1d4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:28.827661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:28.854091 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602da525-16e3-4ab1-95da-f2417904c061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:28.854852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602da525-16e3-4ab1-95da-f2417904c061 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:28.855133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602da525-16e3-4ab1-95da-f2417904c061 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:28.855437 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602da525-16e3-4ab1-95da-f2417904c061 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:28.856100 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602da525-16e3-4ab1-95da-f2417904c061 None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:28.856646 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602da525-16e3-4ab1-95da-f2417904c061 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:28.858880 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602da525-16e3-4ab1-95da-f2417904c061 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:28.865942 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:28.871255 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 285/549] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 12:23:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:23:28.872643 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485815a9-5807-4059-8679-3713870fdc9e None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:28.873200 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485815a9-5807-4059-8679-3713870fdc9e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:28.873627 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485815a9-5807-4059-8679-3713870fdc9e None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:28.874992 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 286/550] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:28.880036 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485815a9-5807-4059-8679-3713870fdc9e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:28.923988 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0147c-73bb-45bc-b241-88c44160b683 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:28.963876 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b78c3b-73dd-4c8a-a762-5f180960ae30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:28.964751 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b78c3b-73dd-4c8a-a762-5f180960ae30 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:28.965181 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b78c3b-73dd-4c8a-a762-5f180960ae30 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:28.965602 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b78c3b-73dd-4c8a-a762-5f180960ae30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:28.978012 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b78c3b-73dd-4c8a-a762-5f180960ae30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:28.980110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b78c3b-73dd-4c8a-a762-5f180960ae30 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:28.996622 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-93b78c3b-73dd-4c8a-a762-5f180960ae30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:28.996622 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:28.996622 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:28.996622 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:28.996622 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:28.996622 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:28.996622 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:28.996622 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:28.996622 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:28.996622 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:28.996622 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:28.996622 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:28.996622 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:28.996622 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:28.996622 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:28.996622 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:28.997686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:28.997686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:28.997686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:28.997686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:28.997686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:28.997686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:28.997686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:28.997686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:28.997686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:28.997686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:28.997686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:28.997686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:28.997686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:28.997686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:28.997686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:28.997686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:28.997686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:28.998575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:28.998575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:28.998575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:29.000634 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 265/551] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:29.002015 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4573588-c622-47e4-b486-304aaa046197 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:29.002482 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4573588-c622-47e4-b486-304aaa046197 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:29.002922 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4573588-c622-47e4-b486-304aaa046197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:29.021235 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485815a9-5807-4059-8679-3713870fdc9e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:29.022663 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b4573588-c622-47e4-b486-304aaa046197 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:29.024126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b4573588-c622-47e4-b486-304aaa046197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:29.067491 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957faa38-d149-45da-bfa2-8a67d3ab9c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:29.068143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957faa38-d149-45da-bfa2-8a67d3ab9c71 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:29.068425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957faa38-d149-45da-bfa2-8a67d3ab9c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:29.069340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957faa38-d149-45da-bfa2-8a67d3ab9c71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:29.085989 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957faa38-d149-45da-bfa2-8a67d3ab9c71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:29.088498 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957faa38-d149-45da-bfa2-8a67d3ab9c71 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:29.103421 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 266/552] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:29.104326 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4573588-c622-47e4-b486-304aaa046197 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:29.107361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-957faa38-d149-45da-bfa2-8a67d3ab9c71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:29.107361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:29.107361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:29.107361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:29.107361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:29.107361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:29.107361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:29.107361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:29.107361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:29.107361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:29.107361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:29.107361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:29.107361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:29.107361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:29.107361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:29.107361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:29.108265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:29.108265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:29.108265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:29.108265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:29.108265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:29.108265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:29.108265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:29.108265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:29.108265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:29.108265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:29.108265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:29.108265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:29.108265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:29.108265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:29.108265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:29.108265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:29.108265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:29.109130 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:29.109130 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:29.109130 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:29.111167 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 287/553] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:29.118494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957faa38-d149-45da-bfa2-8a67d3ab9c71 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:29.156761 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:29.164181 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:29.226279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4573588-c622-47e4-b486-304aaa046197 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:29.254308 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957faa38-d149-45da-bfa2-8a67d3ab9c71 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:29.273140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d6f6e-d074-46d0-8171-39b923ccf09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:29.274092 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d6f6e-d074-46d0-8171-39b923ccf09c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:29.280267 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d6f6e-d074-46d0-8171-39b923ccf09c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:29.280776 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d6f6e-d074-46d0-8171-39b923ccf09c None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:29.281781 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641d6f6e-d074-46d0-8171-39b923ccf09c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:29.282303 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641d6f6e-d074-46d0-8171-39b923ccf09c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:29.284233 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641d6f6e-d074-46d0-8171-39b923ccf09c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:29.296226 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a1330-9359-4736-83e2-73f4064c31b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:29.296849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a1330-9359-4736-83e2-73f4064c31b8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:29.297131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a1330-9359-4736-83e2-73f4064c31b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:29.297410 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a1330-9359-4736-83e2-73f4064c31b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:29.298149 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a1330-9359-4736-83e2-73f4064c31b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:29.302507 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a1330-9359-4736-83e2-73f4064c31b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:29.302507 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a1330-9359-4736-83e2-73f4064c31b8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:29.303189 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 267/554] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:29.304321 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d6f6e-d074-46d0-8171-39b923ccf09c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:29.317347 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 288/555] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:29.317347 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a1330-9359-4736-83e2-73f4064c31b8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:29.427596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:29.430109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:29.435398 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d6f6e-d074-46d0-8171-39b923ccf09c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:29.442178 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a1330-9359-4736-83e2-73f4064c31b8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:29.474027 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d92e8-c5bb-4c9c-917d-265abbc37d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:29.475128 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d92e8-c5bb-4c9c-917d-265abbc37d39 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:29.475702 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d92e8-c5bb-4c9c-917d-265abbc37d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:29.476187 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d92e8-c5bb-4c9c-917d-265abbc37d39 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:29.486130 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ad017-db11-48a8-b03d-d16d8f5e64f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:29.486691 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ad017-db11-48a8-b03d-d16d8f5e64f8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:29.486976 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ad017-db11-48a8-b03d-d16d8f5e64f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:29.487284 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ad017-db11-48a8-b03d-d16d8f5e64f8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:29.488730 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d92e8-c5bb-4c9c-917d-265abbc37d39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:29.495265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-5b9d92e8-c5bb-4c9c-917d-265abbc37d39 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:29.495265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:29.495265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:29.495265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:29.495265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:29.495265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:29.495265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:29.495265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:29.495265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:29.495265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:29.495265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:29.495265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:29.495265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:29.495265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:29.495265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:29.495265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:29.496508 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:29.496508 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:29.496508 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:29.496508 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:29.496508 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:29.496508 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:29.496508 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:29.496508 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:29.496508 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:29.496508 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:29.496508 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:29.496508 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:29.496508 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:29.496508 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:29.496508 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:29.496508 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:29.497327 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:29.497327 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:29.497327 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:29.497327 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:29.497327 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:29.497327 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:29.497327 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:29.497327 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:29.497327 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:29.497327 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:29.497327 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:29.497327 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:29.498028 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ad017-db11-48a8-b03d-d16d8f5e64f8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:29.498578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ad017-db11-48a8-b03d-d16d8f5e64f8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:29.505056 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 268/556] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 12:23:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:29.506748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d92e8-c5bb-4c9c-917d-265abbc37d39 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:29.512265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-b14ad017-db11-48a8-b03d-d16d8f5e64f8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 12:23:29.512265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:29.512265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:29.512265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:29.512265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:29.512265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:29.512265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:29.512265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:29.512265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:29.512265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:29.512265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:29.512265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:29.512265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:23:29.512265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:23:29.512265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:23:29.512265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:23:29.513196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:29.513196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:29.513196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:23:29.513196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:29.513196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:23:29.513196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:29.513196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:29.513196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:29.513196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:29.513196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:29.513196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:29.513196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:29.513196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:29.513196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:29.513196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:29.513196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:29.513196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:29.513196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:29.514579 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:29.514579 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:29.514579 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:23:29.514579 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:23:29.514579 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:23:29.514579 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:23:29.514579 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:23:29.514579 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:23:29.514579 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 12:23:29.514579 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:29.514579 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 289/557] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:23:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:29.514579 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ad017-db11-48a8-b03d-d16d8f5e64f8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:29.618781 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d92e8-c5bb-4c9c-917d-265abbc37d39 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:29.628394 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ad017-db11-48a8-b03d-d16d8f5e64f8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:29.653911 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e0dde-7cc7-4062-9262-eb50f57cb967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:29.654534 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e0dde-7cc7-4062-9262-eb50f57cb967 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:29.654812 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e0dde-7cc7-4062-9262-eb50f57cb967 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:29.655080 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e0dde-7cc7-4062-9262-eb50f57cb967 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:29.663037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e0dde-7cc7-4062-9262-eb50f57cb967 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:29.664568 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e0dde-7cc7-4062-9262-eb50f57cb967 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:29.666919 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d0746a-32b7-45fa-90f5-48846c2c7ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:29.667733 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0746a-32b7-45fa-90f5-48846c2c7ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:29.668159 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0746a-32b7-45fa-90f5-48846c2c7ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:29.668594 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0746a-32b7-45fa-90f5-48846c2c7ad2 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:29.669391 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d0746a-32b7-45fa-90f5-48846c2c7ad2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:29.669836 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d0746a-32b7-45fa-90f5-48846c2c7ad2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:29.671364 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d0746a-32b7-45fa-90f5-48846c2c7ad2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:29.676358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-da5e0dde-7cc7-4062-9262-eb50f57cb967 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 12:23:29.676358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:29.676358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:29.676358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:29.676358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:29.676358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:29.676358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:29.676358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:29.676358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:29.676358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:29.676358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:29.676358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:29.676358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:23:29.676358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:23:29.676358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:23:29.676358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:23:29.677293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:29.677293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:29.677293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:23:29.677293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:29.677293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:23:29.677293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:29.677293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:29.677293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:29.677293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:29.677293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:29.677293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:29.677293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:29.677293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:29.677293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:29.677293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:29.677293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:29.677293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:29.677293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:29.678536 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:29.678536 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:29.678536 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:23:29.678536 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:23:29.678536 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:23:29.678536 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:23:29.678536 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:23:29.678536 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:23:29.678536 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 12:23:29.678536 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:29.678536 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 269/558] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:23:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:29.679422 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5171136-ad11-44ff-bfe9-39c2248055a6 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:29.679422 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5171136-ad11-44ff-bfe9-39c2248055a6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:29.679422 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5171136-ad11-44ff-bfe9-39c2248055a6 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:29.680515 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 270/559] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:29.685005 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5171136-ad11-44ff-bfe9-39c2248055a6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:29.685494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 290/560] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 12:23:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:29.688396 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d0746a-32b7-45fa-90f5-48846c2c7ad2 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:29.703914 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:29.705288 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:29.792534 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5171136-ad11-44ff-bfe9-39c2248055a6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:29.794959 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d0746a-32b7-45fa-90f5-48846c2c7ad2 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:29.825916 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756fce6-1dae-4acb-85b2-e4d77027c379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:29.826513 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756fce6-1dae-4acb-85b2-e4d77027c379 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:29.826801 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756fce6-1dae-4acb-85b2-e4d77027c379 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:29.827072 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756fce6-1dae-4acb-85b2-e4d77027c379 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:29.827819 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7756fce6-1dae-4acb-85b2-e4d77027c379 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:29.828024 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73428e49-4ead-4ad7-a197-0418281433b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:29.828128 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7756fce6-1dae-4acb-85b2-e4d77027c379 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:29.828628 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73428e49-4ead-4ad7-a197-0418281433b8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:29.828904 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73428e49-4ead-4ad7-a197-0418281433b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:29.829171 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73428e49-4ead-4ad7-a197-0418281433b8 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:29.829551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7756fce6-1dae-4acb-85b2-e4d77027c379 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:29.829877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73428e49-4ead-4ad7-a197-0418281433b8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:29.830178 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73428e49-4ead-4ad7-a197-0418281433b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:29.831594 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73428e49-4ead-4ad7-a197-0418281433b8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:29.848332 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 271/561] 10.222.0.49 () {56 vars in 1135 bytes} [Wed Aug 10 12:23:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:29.854122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 291/562] 10.222.0.49 () {56 vars in 1137 bytes} [Wed Aug 10 12:23:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:29.854264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d51f4-f2d5-433c-8c3a-91cde8562d8a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:29.854464 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73428e49-4ead-4ad7-a197-0418281433b8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:29.854550 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d51f4-f2d5-433c-8c3a-91cde8562d8a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:29.854550 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d51f4-f2d5-433c-8c3a-91cde8562d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:29.869235 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b44d51f4-f2d5-433c-8c3a-91cde8562d8a None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:29.870676 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b44d51f4-f2d5-433c-8c3a-91cde8562d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:29.920101 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:29.937251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 272/563] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:29.938483 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc18cf-134b-4f15-b374-e9ac38bd1398 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:29.938764 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc18cf-134b-4f15-b374-e9ac38bd1398 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:29.939072 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc18cf-134b-4f15-b374-e9ac38bd1398 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:29.943220 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 273/564] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:29.943220 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc18cf-134b-4f15-b374-e9ac38bd1398 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:29.950864 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:29.957847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73428e49-4ead-4ad7-a197-0418281433b8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:29.993163 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ade6a2-3849-46d0-b1b3-1ba929426202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:29.993794 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ade6a2-3849-46d0-b1b3-1ba929426202 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:29.994076 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ade6a2-3849-46d0-b1b3-1ba929426202 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:29.994351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ade6a2-3849-46d0-b1b3-1ba929426202 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:30.004891 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ade6a2-3849-46d0-b1b3-1ba929426202 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:30.006987 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ade6a2-3849-46d0-b1b3-1ba929426202 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:30.021155 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-73ade6a2-3849-46d0-b1b3-1ba929426202 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:30.021155 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:30.021155 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:30.021155 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:30.021155 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:30.021155 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:30.021155 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:30.021155 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:30.021155 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:30.021155 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:30.021155 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:30.021155 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:30.021155 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:30.021155 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:30.021155 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:30.021155 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:30.022123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:30.022123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:30.022123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:30.022123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:30.022123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:30.022123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:30.022123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:30.022123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:30.022123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:30.022123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:30.022123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:30.022123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:30.022123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:30.022123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:30.022123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:30.022123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:30.022123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:30.023008 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:30.023008 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:30.023008 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:30.023008 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 292/565] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:30.023292 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ade6a2-3849-46d0-b1b3-1ba929426202 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:30.047058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc18cf-134b-4f15-b374-e9ac38bd1398 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:30.081367 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8353aae-d2d3-479c-aa8e-80fbea66bf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:30.082065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8353aae-d2d3-479c-aa8e-80fbea66bf03 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:30.082342 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8353aae-d2d3-479c-aa8e-80fbea66bf03 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:30.082616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8353aae-d2d3-479c-aa8e-80fbea66bf03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:30.123008 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ade6a2-3849-46d0-b1b3-1ba929426202 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:30.142660 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:30.150449 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8353aae-d2d3-479c-aa8e-80fbea66bf03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:30.151941 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8353aae-d2d3-479c-aa8e-80fbea66bf03 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:30.157522 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de92aa-45bc-4686-aa75-77b0b77335bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:30.158153 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de92aa-45bc-4686-aa75-77b0b77335bb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:30.158435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de92aa-45bc-4686-aa75-77b0b77335bb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:30.158712 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de92aa-45bc-4686-aa75-77b0b77335bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:30.159529 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:30.169385 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de92aa-45bc-4686-aa75-77b0b77335bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:30.170823 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de92aa-45bc-4686-aa75-77b0b77335bb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:30.184486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-62de92aa-45bc-4686-aa75-77b0b77335bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:30.184486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:30.184486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:30.184486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:30.184486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:30.184486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:30.184486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:30.184486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:30.184486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:30.184486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:30.184486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:30.184486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:30.184486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:30.184486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:30.184486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:30.184486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:30.185565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:30.185565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:30.185565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:30.185565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:30.185565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:30.185565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:30.185565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:30.185565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:30.185565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:30.185565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:30.185565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:30.185565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:30.185565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:30.185565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:30.185565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:30.185565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:30.185565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:30.186504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:30.186504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:30.186504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:30.186504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 293/566] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:30.187833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4b53f-dc2d-4882-8dca-67843c183cd5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:30.188254 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4b53f-dc2d-4882-8dca-67843c183cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:30.188823 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4b53f-dc2d-4882-8dca-67843c183cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:30.206589 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e1e4b53f-dc2d-4882-8dca-67843c183cd5 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:30.207818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e1e4b53f-dc2d-4882-8dca-67843c183cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:30.251385 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 274/567] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:23:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:23:30.252439 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8353aae-d2d3-479c-aa8e-80fbea66bf03 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:30.283053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 294/568] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:30.283053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e4b53f-dc2d-4882-8dca-67843c183cd5 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:30.355496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8353aae-d2d3-479c-aa8e-80fbea66bf03 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:30.362587 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:30.379500 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e4b53f-dc2d-4882-8dca-67843c183cd5 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:30.383318 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:30.389929 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535375f-a7cd-4d5f-96ef-08c1559e10a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:30.390553 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535375f-a7cd-4d5f-96ef-08c1559e10a7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:30.390841 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535375f-a7cd-4d5f-96ef-08c1559e10a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:30.391114 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535375f-a7cd-4d5f-96ef-08c1559e10a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:30.401138 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a535375f-a7cd-4d5f-96ef-08c1559e10a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:30.405822 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a535375f-a7cd-4d5f-96ef-08c1559e10a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:30.405822 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:30.405822 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:30.405822 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:30.405822 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:30.405822 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:30.405822 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:30.405822 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:30.405822 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:30.405822 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:30.405822 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:30.405822 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:30.405822 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:30.405822 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:30.405822 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:30.405822 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:30.406900 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:30.406900 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:30.406900 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:30.406900 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:30.406900 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:30.406900 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:30.406900 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:30.406900 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:30.406900 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:30.406900 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:30.406900 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:30.406900 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:30.406900 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:30.406900 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:30.406900 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:30.406900 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:30.407856 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:30.407856 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:30.407856 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:30.407856 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:30.407856 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:30.407856 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:30.407856 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:30.407856 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:30.407856 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:30.407856 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:30.407856 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:30.407856 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:30.408821 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 275/569] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 12:23:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:30.409735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535375f-a7cd-4d5f-96ef-08c1559e10a7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:30.413793 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92e7e6-5199-41f2-ae5c-f29511714fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:30.414450 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92e7e6-5199-41f2-ae5c-f29511714fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:30.414729 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92e7e6-5199-41f2-ae5c-f29511714fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:30.414998 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92e7e6-5199-41f2-ae5c-f29511714fc9 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:30.415665 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92e7e6-5199-41f2-ae5c-f29511714fc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:30.415971 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92e7e6-5199-41f2-ae5c-f29511714fc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:30.417340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92e7e6-5199-41f2-ae5c-f29511714fc9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:30.428995 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 295/570] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:30.429809 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92e7e6-5199-41f2-ae5c-f29511714fc9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:30.510178 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535375f-a7cd-4d5f-96ef-08c1559e10a7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:30.529853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92e7e6-5199-41f2-ae5c-f29511714fc9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:30.544062 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf8ae5-1dae-4133-abec-cc387560d582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:30.544664 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf8ae5-1dae-4133-abec-cc387560d582 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:30.544951 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf8ae5-1dae-4133-abec-cc387560d582 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:30.545226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf8ae5-1dae-4133-abec-cc387560d582 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:30.545866 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf8ae5-1dae-4133-abec-cc387560d582 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:30.546170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf8ae5-1dae-4133-abec-cc387560d582 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:30.547571 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf8ae5-1dae-4133-abec-cc387560d582 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:30.558587 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 276/571] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:30.559424 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf8ae5-1dae-4133-abec-cc387560d582 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:30.566268 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6becf-8ac2-46e9-bf58-7f66190f545f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:30.566902 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6becf-8ac2-46e9-bf58-7f66190f545f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:30.567202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6becf-8ac2-46e9-bf58-7f66190f545f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:30.567479 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6becf-8ac2-46e9-bf58-7f66190f545f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:30.578181 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6becf-8ac2-46e9-bf58-7f66190f545f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:30.579643 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6becf-8ac2-46e9-bf58-7f66190f545f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:30.586369 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:30.593928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-4dc6becf-8ac2-46e9-bf58-7f66190f545f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:30.593928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:30.593928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:30.593928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:30.593928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:30.593928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:30.593928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:30.593928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:30.593928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:30.593928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:30.593928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:30.593928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:30.593928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:30.593928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:30.593928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:30.593928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:30.594894 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:30.594894 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:30.594894 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:30.594894 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:30.594894 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:30.594894 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:30.594894 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:30.594894 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:30.594894 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:30.594894 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:30.594894 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:30.594894 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:30.594894 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:30.594894 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:30.594894 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:30.594894 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:30.594894 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:30.600797 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:30.600797 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:30.600797 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:30.600797 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 296/572] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:30.601484 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6becf-8ac2-46e9-bf58-7f66190f545f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:30.617390 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:30.658077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf8ae5-1dae-4133-abec-cc387560d582 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:30.691727 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c8db4-b9ae-43ec-8304-823786845925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:30.692349 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c8db4-b9ae-43ec-8304-823786845925 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:30.692663 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c8db4-b9ae-43ec-8304-823786845925 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:30.692936 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c8db4-b9ae-43ec-8304-823786845925 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:30.699366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6becf-8ac2-46e9-bf58-7f66190f545f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:30.703435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713c8db4-b9ae-43ec-8304-823786845925 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:30.708086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-713c8db4-b9ae-43ec-8304-823786845925 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:30.708086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:30.708086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:30.708086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:30.708086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:30.708086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:30.708086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:30.708086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:30.708086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:30.708086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:30.708086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:30.708086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:30.708086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:30.708086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:30.708086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:30.708086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:30.709074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:30.709074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:30.709074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:30.709074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:30.709074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:30.709074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:30.709074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:30.709074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:30.709074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:30.709074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:30.709074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:30.709074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:30.709074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:30.709074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:30.709074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:30.709074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:30.715470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:30.715470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:30.715470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:30.715470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:30.715470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:30.715470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:30.715470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:30.715470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:30.715470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:30.715470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:30.715470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:30.715470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:30.715470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 277/573] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 12:23:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:30.715470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c8db4-b9ae-43ec-8304-823786845925 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:30.733625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e1d96-6015-4afe-8d76-066f5f59fd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:30.734227 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e1d96-6015-4afe-8d76-066f5f59fd38 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:30.734504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e1d96-6015-4afe-8d76-066f5f59fd38 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:30.734775 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e1d96-6015-4afe-8d76-066f5f59fd38 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:30.735459 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e1d96-6015-4afe-8d76-066f5f59fd38 None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:30.735763 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e1d96-6015-4afe-8d76-066f5f59fd38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:30.737112 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e1d96-6015-4afe-8d76-066f5f59fd38 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:30.744860 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 297/574] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 12:23:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:30.745895 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e1d96-6015-4afe-8d76-066f5f59fd38 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:30.801377 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:30.816835 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c8db4-b9ae-43ec-8304-823786845925 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:30.825855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:30.847875 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e1d96-6015-4afe-8d76-066f5f59fd38 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:30.852052 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6320c4-551d-479b-b339-087fa1eba920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:30.852760 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6320c4-551d-479b-b339-087fa1eba920 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:30.853042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6320c4-551d-479b-b339-087fa1eba920 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:30.853317 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6320c4-551d-479b-b339-087fa1eba920 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:30.860408 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6320c4-551d-479b-b339-087fa1eba920 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:30.861856 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6320c4-551d-479b-b339-087fa1eba920 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:30.873555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-0c6320c4-551d-479b-b339-087fa1eba920 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:23:30.873555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:30.873555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:30.873555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:30.873555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:30.873555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:30.873555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:30.873555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:30.873555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:30.873555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:30.873555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:30.873555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:30.873555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:23:30.873555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:23:30.873555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:23:30.873555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:23:30.874601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:30.874601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:30.874601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:23:30.874601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:30.874601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:23:30.874601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:30.874601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:30.874601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:30.874601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:30.874601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:30.874601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:30.874601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:30.874601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:30.874601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:30.874601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:30.874601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:30.874601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:30.874601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:30.880507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:30.880507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:30.880507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:23:30.880507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:23:30.880507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:23:30.880507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:23:30.880507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:23:30.880507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:23:30.880507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:23:30.880507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:30.880507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 278/575] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Aug 10 12:23:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:30.880507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6320c4-551d-479b-b339-087fa1eba920 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:30.882741 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19def1e6-4221-47f6-918f-ccee47f40743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:30.883402 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19def1e6-4221-47f6-918f-ccee47f40743 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:30.883680 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19def1e6-4221-47f6-918f-ccee47f40743 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:30.883950 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19def1e6-4221-47f6-918f-ccee47f40743 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:30.890672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19def1e6-4221-47f6-918f-ccee47f40743 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:30.892170 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19def1e6-4221-47f6-918f-ccee47f40743 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:30.904047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-19def1e6-4221-47f6-918f-ccee47f40743 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:30.904047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:30.904047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:30.904047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:30.904047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:30.904047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:30.904047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:30.904047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:30.904047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:30.904047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:30.904047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:30.904047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:30.904047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:23:30.904047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:23:30.904047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:23:30.905081 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:23:30.905081 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:30.905081 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:30.905081 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:30.905081 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:30.905081 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:30.905081 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:30.905081 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:30.905081 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:30.905081 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:30.905081 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:30.905081 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:30.905081 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:30.905081 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:30.905081 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:30.905081 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:30.905081 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:30.906141 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:23:30.906141 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:23:30.906141 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:23:30.906141 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:23:30.906141 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:23:30.906141 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:23:30.906141 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:30.906141 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:30.909801 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 298/576] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:30.910465 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19def1e6-4221-47f6-918f-ccee47f40743 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:30.975094 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6320c4-551d-479b-b339-087fa1eba920 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:31.043343 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19def1e6-4221-47f6-918f-ccee47f40743 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:31.044742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a6c77a-a683-4c26-b855-2ae2d2fa48bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:31.045342 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a6c77a-a683-4c26-b855-2ae2d2fa48bd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:31.045620 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a6c77a-a683-4c26-b855-2ae2d2fa48bd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:31.045892 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a6c77a-a683-4c26-b855-2ae2d2fa48bd None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:31.046523 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a6c77a-a683-4c26-b855-2ae2d2fa48bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:31.046825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a6c77a-a683-4c26-b855-2ae2d2fa48bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:31.048214 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a6c77a-a683-4c26-b855-2ae2d2fa48bd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:31.060616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 279/577] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:31.061236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a6c77a-a683-4c26-b855-2ae2d2fa48bd None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:31.071495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:31.082069 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff6d47-05d9-4360-9226-76cb660de806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:31.085321 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff6d47-05d9-4360-9226-76cb660de806 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:31.085321 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff6d47-05d9-4360-9226-76cb660de806 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:31.085321 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff6d47-05d9-4360-9226-76cb660de806 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:31.085321 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff6d47-05d9-4360-9226-76cb660de806 None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:31.085321 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff6d47-05d9-4360-9226-76cb660de806 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:31.085747 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff6d47-05d9-4360-9226-76cb660de806 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:31.087759 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:31.093889 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 299/578] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 12:23:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:31.094662 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff6d47-05d9-4360-9226-76cb660de806 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:31.163621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a6c77a-a683-4c26-b855-2ae2d2fa48bd None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:31.194514 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff6d47-05d9-4360-9226-76cb660de806 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:31.197212 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3deecb-d8a1-4692-942d-69f389b92caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:31.197836 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3deecb-d8a1-4692-942d-69f389b92caa None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:31.198111 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3deecb-d8a1-4692-942d-69f389b92caa None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:31.198382 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3deecb-d8a1-4692-942d-69f389b92caa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:31.209222 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3deecb-d8a1-4692-942d-69f389b92caa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:31.211104 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3deecb-d8a1-4692-942d-69f389b92caa None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:31.224746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-8f3deecb-d8a1-4692-942d-69f389b92caa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:31.224746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:31.224746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:31.224746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:31.224746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:31.224746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:31.224746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:31.224746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:31.224746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:31.224746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:31.224746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:31.224746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:31.224746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:31.224746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:31.224746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:31.224746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:31.225860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:31.225860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:31.225860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:31.225860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:31.225860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:31.225860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:31.225860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:31.225860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:31.225860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:31.225860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:31.225860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:31.225860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:31.225860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:31.225860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:31.225860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:31.225860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:31.225860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:31.226775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:31.226775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:31.226775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:31.226775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 280/579] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:31.226775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3deecb-d8a1-4692-942d-69f389b92caa None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:31.232935 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a43251-1ddc-4b66-83d4-4b01e4080736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:31.233557 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a43251-1ddc-4b66-83d4-4b01e4080736 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:31.233836 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a43251-1ddc-4b66-83d4-4b01e4080736 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:31.234110 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a43251-1ddc-4b66-83d4-4b01e4080736 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:31.234835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a43251-1ddc-4b66-83d4-4b01e4080736 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:31.235144 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a43251-1ddc-4b66-83d4-4b01e4080736 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:31.236590 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a43251-1ddc-4b66-83d4-4b01e4080736 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:31.254346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 300/580] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Aug 10 12:23:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:31.255146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a43251-1ddc-4b66-83d4-4b01e4080736 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:31.288353 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:31.310305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:31.331619 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3deecb-d8a1-4692-942d-69f389b92caa None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:31.353765 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a43251-1ddc-4b66-83d4-4b01e4080736 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:31.364362 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40cf476-72bb-4519-bd27-0cded08b9d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:31.364972 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40cf476-72bb-4519-bd27-0cded08b9d21 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:31.365253 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40cf476-72bb-4519-bd27-0cded08b9d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:31.366078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40cf476-72bb-4519-bd27-0cded08b9d21 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:31.366953 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40cf476-72bb-4519-bd27-0cded08b9d21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:31.367286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40cf476-72bb-4519-bd27-0cded08b9d21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:31.368790 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40cf476-72bb-4519-bd27-0cded08b9d21 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:31.381088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 281/581] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:23:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:31.382014 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40cf476-72bb-4519-bd27-0cded08b9d21 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:31.387848 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077f5cc-0c14-496e-9e39-e49cb2108bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:31.388480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077f5cc-0c14-496e-9e39-e49cb2108bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:31.388753 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077f5cc-0c14-496e-9e39-e49cb2108bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:31.389023 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077f5cc-0c14-496e-9e39-e49cb2108bc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:31.395694 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077f5cc-0c14-496e-9e39-e49cb2108bc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:31.397645 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077f5cc-0c14-496e-9e39-e49cb2108bc7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:31.408432 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-6077f5cc-0c14-496e-9e39-e49cb2108bc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:31.408432 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:31.408432 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:31.408432 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:31.408432 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:31.408432 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:31.408432 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:31.408432 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:31.408432 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:31.408432 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:31.408432 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:31.408432 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:31.408432 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:23:31.408432 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:23:31.408432 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:23:31.409326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:23:31.409326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:31.409326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:31.409326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:31.409326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:31.409326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:31.409326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:31.409326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:31.409326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:31.409326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:31.409326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:31.409326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:31.409326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:31.409326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:31.409326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:31.409326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:31.409326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:31.410210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:23:31.410210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:23:31.410210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:23:31.410210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:23:31.410210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:23:31.410210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:23:31.410210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:31.410210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:31.410210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 301/582] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:31.410210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077f5cc-0c14-496e-9e39-e49cb2108bc7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:31.486141 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40cf476-72bb-4519-bd27-0cded08b9d21 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:31.504167 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:31.510052 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077f5cc-0c14-496e-9e39-e49cb2108bc7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:31.521173 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa95be-1bdc-4129-a711-b789dfb94109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:31.522666 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa95be-1bdc-4129-a711-b789dfb94109 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:31.522963 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa95be-1bdc-4129-a711-b789dfb94109 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:31.523278 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa95be-1bdc-4129-a711-b789dfb94109 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:31.527674 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:31.533290 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa95be-1bdc-4129-a711-b789dfb94109 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:31.535101 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa95be-1bdc-4129-a711-b789dfb94109 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:31.545148 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26aac7-7c04-4585-9287-89c06253af1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:31.545751 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26aac7-7c04-4585-9287-89c06253af1b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:31.546033 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26aac7-7c04-4585-9287-89c06253af1b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:31.546307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26aac7-7c04-4585-9287-89c06253af1b None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:31.546940 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a26aac7-7c04-4585-9287-89c06253af1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:31.547280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a26aac7-7c04-4585-9287-89c06253af1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:31.548710 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a26aac7-7c04-4585-9287-89c06253af1b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:31.550118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-51fa95be-1bdc-4129-a711-b789dfb94109 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:31.550118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:31.550118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:31.550118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:31.550118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:31.550118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:31.550118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:31.550118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:31.550118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:31.550118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:31.550118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:31.550118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:31.550118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:31.550118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:31.550118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:31.550118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:31.551078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:31.551078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:31.551078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:31.551078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:31.551078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:31.551078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:31.551078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:31.551078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:31.551078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:31.551078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:31.551078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:31.551078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:31.551078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:31.551078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:31.551078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:31.551078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:31.551078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:31.552225 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:31.552225 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:31.552225 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:31.552225 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 282/583] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:31.552225 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa95be-1bdc-4129-a711-b789dfb94109 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:31.560625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 302/584] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:31.561486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26aac7-7c04-4585-9287-89c06253af1b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:31.652983 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa95be-1bdc-4129-a711-b789dfb94109 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:31.660033 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26aac7-7c04-4585-9287-89c06253af1b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:31.685956 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9cb46c-677d-40fc-932b-edcc33c789bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:31.686586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9cb46c-677d-40fc-932b-edcc33c789bb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:31.686872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9cb46c-677d-40fc-932b-edcc33c789bb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:31.687150 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9cb46c-677d-40fc-932b-edcc33c789bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:31.693121 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84a5e3-8d78-4d4d-aaee-63a9be89f72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:31.694403 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84a5e3-8d78-4d4d-aaee-63a9be89f72e None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:31.694796 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84a5e3-8d78-4d4d-aaee-63a9be89f72e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:31.695074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84a5e3-8d78-4d4d-aaee-63a9be89f72e None admin] PATH_INFO: `/v3/projects/8e201e5f281a4977be503e22daf149d7/users/ab5a0d9c39d84cc7a91f5a752dcf2b70/roles/a9bbb011afca4176a3def9356cb5f165` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:31.698479 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9cb46c-677d-40fc-932b-edcc33c789bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:31.699967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9cb46c-677d-40fc-932b-edcc33c789bb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:31.710200 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84a5e3-8d78-4d4d-aaee-63a9be89f72e None admin] RBAC: Authorizing `identity:create_grant(project_id=8e201e5f281a4977be503e22daf149d7, user_id=ab5a0d9c39d84cc7a91f5a752dcf2b70, role_id=a9bbb011afca4176a3def9356cb5f165)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:31.711640 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84a5e3-8d78-4d4d-aaee-63a9be89f72e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:31.714058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-4f9cb46c-677d-40fc-932b-edcc33c789bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:31.714058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:31.714058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:31.714058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:31.714058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:31.714058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:31.714058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:31.714058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:31.714058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:31.714058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:31.714058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:31.714058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:31.714058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:31.714058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:31.714058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:31.714058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:31.715090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:31.715090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:31.715090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:31.715090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:31.715090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:31.715090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:31.715090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:31.715090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:31.715090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:31.715090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:31.715090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:31.715090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:31.715090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:31.715090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:31.715090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:31.715090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:31.715090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:31.716022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:31.716022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:31.716022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:31.716022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 283/585] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:31.716022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9cb46c-677d-40fc-932b-edcc33c789bb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:31.740522 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:31.779411 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 303/586] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 12:23:31 2022] PUT /identity/v3/projects/8e201e5f281a4977be503e22daf149d7/users/ab5a0d9c39d84cc7a91f5a752dcf2b70/roles/a9bbb011afca4176a3def9356cb5f165 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:23:31.780304 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84a5e3-8d78-4d4d-aaee-63a9be89f72e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:31.822051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:31.825153 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9cb46c-677d-40fc-932b-edcc33c789bb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:31.867705 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad4fd0-c081-4e9c-be3a-084a3f3daacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:31.868328 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad4fd0-c081-4e9c-be3a-084a3f3daacf None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:31.868627 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad4fd0-c081-4e9c-be3a-084a3f3daacf None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:31.868897 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad4fd0-c081-4e9c-be3a-084a3f3daacf None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:31.869613 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad4fd0-c081-4e9c-be3a-084a3f3daacf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:31.869904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad4fd0-c081-4e9c-be3a-084a3f3daacf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:31.871684 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad4fd0-c081-4e9c-be3a-084a3f3daacf None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:31.884885 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 284/587] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:23:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:31.885800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad4fd0-c081-4e9c-be3a-084a3f3daacf None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:31.913791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84a5e3-8d78-4d4d-aaee-63a9be89f72e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:31.952586 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b6ad6-314d-4ab6-9b24-3fa7079d0012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:31.953463 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b6ad6-314d-4ab6-9b24-3fa7079d0012 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:31.953894 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b6ad6-314d-4ab6-9b24-3fa7079d0012 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:31.954166 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b6ad6-314d-4ab6-9b24-3fa7079d0012 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:31.954804 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b6ad6-314d-4ab6-9b24-3fa7079d0012 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:31.955268 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b6ad6-314d-4ab6-9b24-3fa7079d0012 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:31.956623 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b6ad6-314d-4ab6-9b24-3fa7079d0012 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:31.968891 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 304/588] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:31.969725 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b6ad6-314d-4ab6-9b24-3fa7079d0012 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:31.988068 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:32.008412 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad4fd0-c081-4e9c-be3a-084a3f3daacf None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:32.044906 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3850f01c-b306-42c6-b3b9-bd8aebfc27dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:32.045544 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850f01c-b306-42c6-b3b9-bd8aebfc27dd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:32.045821 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850f01c-b306-42c6-b3b9-bd8aebfc27dd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:32.046097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850f01c-b306-42c6-b3b9-bd8aebfc27dd None admin] PATH_INFO: `/v3/users/glance` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:32.053709 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3850f01c-b306-42c6-b3b9-bd8aebfc27dd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:32.055138 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3850f01c-b306-42c6-b3b9-bd8aebfc27dd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:32.062738 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:32.066404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-3850f01c-b306-42c6-b3b9-bd8aebfc27dd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 12:23:32.066404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:32.066404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:32.066404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:32.066404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:32.066404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:32.066404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:32.066404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:32.066404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:32.066404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:32.066404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:32.066404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:32.066404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:23:32.066404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:23:32.066404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:23:32.066404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:23:32.067386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:32.067386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:32.067386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:23:32.067386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:32.067386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:23:32.067386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:32.067386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:32.067386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:32.067386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:32.067386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:32.067386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:32.067386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:32.067386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:32.067386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:32.067386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:32.067386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:32.067386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:32.067386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:32.068564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:32.068564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:32.068564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:23:32.068564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:23:32.068564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:23:32.068564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:23:32.068564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:23:32.068564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:23:32.068564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 12:23:32.068564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:32.073255 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 285/589] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:23:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:32.075693 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3850f01c-b306-42c6-b3b9-bd8aebfc27dd None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:32.082019 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b6ad6-314d-4ab6-9b24-3fa7079d0012 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:32.118066 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a1fca-cdfd-4c5a-96d0-c1650c28e94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:32.118689 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a1fca-cdfd-4c5a-96d0-c1650c28e94d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:32.118956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a1fca-cdfd-4c5a-96d0-c1650c28e94d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:32.119262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a1fca-cdfd-4c5a-96d0-c1650c28e94d None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:32.119904 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619a1fca-cdfd-4c5a-96d0-c1650c28e94d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:32.120212 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619a1fca-cdfd-4c5a-96d0-c1650c28e94d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:32.121828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619a1fca-cdfd-4c5a-96d0-c1650c28e94d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:32.137929 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 305/590] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:32.137929 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a1fca-cdfd-4c5a-96d0-c1650c28e94d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:32.190312 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3850f01c-b306-42c6-b3b9-bd8aebfc27dd None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:32.227911 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d494765-82f6-40d7-a280-5403e2d4d3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:32.228654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d494765-82f6-40d7-a280-5403e2d4d3b1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:32.229057 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d494765-82f6-40d7-a280-5403e2d4d3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:32.229341 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d494765-82f6-40d7-a280-5403e2d4d3b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:32.230173 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d494765-82f6-40d7-a280-5403e2d4d3b1 None admin] RBAC: Adding query filter params (role.id=a9bbb011afca4176a3def9356cb5f165, scope.project.id=8e201e5f281a4977be503e22daf149d7, user.id=59395bfa505c4540b3957deee78eb698) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:32.230487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d494765-82f6-40d7-a280-5403e2d4d3b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:32.232074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d494765-82f6-40d7-a280-5403e2d4d3b1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:32.247218 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 286/591] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:23:32 2022] GET /identity/v3/role_assignments?user.id=59395bfa505c4540b3957deee78eb698&scope.project.id=8e201e5f281a4977be503e22daf149d7&role.id=a9bbb011afca4176a3def9356cb5f165 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:32.247218 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d494765-82f6-40d7-a280-5403e2d4d3b1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:32.247540 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a1fca-cdfd-4c5a-96d0-c1650c28e94d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:32.264463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:32.297520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d03c4b-1c26-4adc-a6c2-62f2d40ade34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:32.298280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d03c4b-1c26-4adc-a6c2-62f2d40ade34 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:32.298629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d03c4b-1c26-4adc-a6c2-62f2d40ade34 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:32.298972 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d03c4b-1c26-4adc-a6c2-62f2d40ade34 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:32.311392 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d03c4b-1c26-4adc-a6c2-62f2d40ade34 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:32.313511 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d03c4b-1c26-4adc-a6c2-62f2d40ade34 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:32.331238 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-47d03c4b-1c26-4adc-a6c2-62f2d40ade34 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 12:23:32.331238 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:32.331238 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:32.331238 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:32.331238 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:32.331238 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:32.331238 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:32.331238 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:32.331238 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:32.331238 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:32.331238 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:32.331238 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:32.331238 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:23:32.331238 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:23:32.331238 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:23:32.331238 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:23:32.332262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:32.332262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:32.332262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:23:32.332262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:32.332262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:23:32.332262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:32.332262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:32.332262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:32.332262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:32.332262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:32.332262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:32.332262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:32.332262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:32.332262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:32.332262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:32.332262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:32.332262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:32.332262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:32.333228 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:32.333228 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:32.333228 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:23:32.333228 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:23:32.333228 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:23:32.333228 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:23:32.333228 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:23:32.333228 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:23:32.333228 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 12:23:32.333228 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:32.335763 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 306/592] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 12:23:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:32.336219 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d03c4b-1c26-4adc-a6c2-62f2d40ade34 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:32.355343 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:32.377977 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d494765-82f6-40d7-a280-5403e2d4d3b1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:32.416902 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b51387-5b33-483b-bac7-34a310db06f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:32.417554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b51387-5b33-483b-bac7-34a310db06f1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:32.417836 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b51387-5b33-483b-bac7-34a310db06f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:32.418127 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b51387-5b33-483b-bac7-34a310db06f1 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:32.418854 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b51387-5b33-483b-bac7-34a310db06f1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:32.419167 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b51387-5b33-483b-bac7-34a310db06f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:32.420651 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b51387-5b33-483b-bac7-34a310db06f1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:32.446231 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 287/593] 10.222.0.49 () {56 vars in 1137 bytes} [Wed Aug 10 12:23:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:32.446231 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b51387-5b33-483b-bac7-34a310db06f1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:32.479799 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d03c4b-1c26-4adc-a6c2-62f2d40ade34 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:32.516301 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:32.518693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd4143-a423-4b29-94e8-71ad42577826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:32.519612 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd4143-a423-4b29-94e8-71ad42577826 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:32.520109 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd4143-a423-4b29-94e8-71ad42577826 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:32.520671 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd4143-a423-4b29-94e8-71ad42577826 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:32.529949 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd4143-a423-4b29-94e8-71ad42577826 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:32.532140 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd4143-a423-4b29-94e8-71ad42577826 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:32.547811 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-2fcd4143-a423-4b29-94e8-71ad42577826 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:32.547811 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:32.547811 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:32.547811 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:32.547811 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:32.547811 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:32.547811 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:32.547811 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:32.547811 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:32.547811 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:32.547811 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:32.547811 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:32.547811 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:23:32.547811 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:23:32.547811 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:23:32.548924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:23:32.548924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:32.548924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:32.548924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:32.548924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:32.548924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:32.548924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:32.548924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:32.548924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:32.548924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:32.548924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:32.548924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:32.548924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:32.548924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:32.548924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:32.548924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:32.548924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:32.550035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:23:32.550035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:23:32.550035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:23:32.550035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:23:32.550035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:23:32.550035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:23:32.550035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:32.550035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:32.554156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 307/594] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:32.555859 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd4143-a423-4b29-94e8-71ad42577826 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:32.561653 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b51387-5b33-483b-bac7-34a310db06f1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:32.605549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180197e0-1249-4724-b29c-4fc3835aced2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:32.606473 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180197e0-1249-4724-b29c-4fc3835aced2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:32.606853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180197e0-1249-4724-b29c-4fc3835aced2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:32.608101 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180197e0-1249-4724-b29c-4fc3835aced2 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:32.608969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180197e0-1249-4724-b29c-4fc3835aced2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:32.609453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180197e0-1249-4724-b29c-4fc3835aced2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:32.610999 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180197e0-1249-4724-b29c-4fc3835aced2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:32.635885 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 288/595] 10.222.0.49 () {56 vars in 1139 bytes} [Wed Aug 10 12:23:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:32.637093 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180197e0-1249-4724-b29c-4fc3835aced2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:32.645028 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:32.687513 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd4143-a423-4b29-94e8-71ad42577826 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:32.724473 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5658e4-fab1-48e6-b8b5-fcfa1c03d474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:32.725948 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5658e4-fab1-48e6-b8b5-fcfa1c03d474 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:32.726545 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5658e4-fab1-48e6-b8b5-fcfa1c03d474 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:32.726951 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5658e4-fab1-48e6-b8b5-fcfa1c03d474 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:32.738744 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5658e4-fab1-48e6-b8b5-fcfa1c03d474 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:32.740553 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5658e4-fab1-48e6-b8b5-fcfa1c03d474 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:32.750355 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180197e0-1249-4724-b29c-4fc3835aced2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:32.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-da5658e4-fab1-48e6-b8b5-fcfa1c03d474 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:32.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:32.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:32.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:32.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:32.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:32.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:32.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:32.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:32.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:32.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:32.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:32.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:32.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:32.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:32.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:32.758762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:32.758762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:32.758762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:32.758762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:32.758762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:32.758762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:32.758762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:32.758762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:32.758762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:32.758762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:32.758762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:32.758762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:32.758762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:32.758762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:32.758762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:32.758762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:32.758762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:32.759673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:32.759673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:32.759673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:32.759673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 308/596] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:32.760479 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5658e4-fab1-48e6-b8b5-fcfa1c03d474 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:32.781167 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:32.791465 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf0328-ac60-4a6b-898d-45fae419b2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:32.796207 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf0328-ac60-4a6b-898d-45fae419b2b8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:32.796207 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf0328-ac60-4a6b-898d-45fae419b2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:32.796207 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf0328-ac60-4a6b-898d-45fae419b2b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:32.796207 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf0328-ac60-4a6b-898d-45fae419b2b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:32.796207 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf0328-ac60-4a6b-898d-45fae419b2b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:32.796207 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf0328-ac60-4a6b-898d-45fae419b2b8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:32.807608 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 289/597] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:23:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:32.808437 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf0328-ac60-4a6b-898d-45fae419b2b8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:32.877287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5658e4-fab1-48e6-b8b5-fcfa1c03d474 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:32.900028 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:32.918885 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37eebe-0067-4a2d-b2e4-b8288151f243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:32.921070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37eebe-0067-4a2d-b2e4-b8288151f243 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:32.921070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37eebe-0067-4a2d-b2e4-b8288151f243 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:32.921070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37eebe-0067-4a2d-b2e4-b8288151f243 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:32.931957 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37eebe-0067-4a2d-b2e4-b8288151f243 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:32.933734 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37eebe-0067-4a2d-b2e4-b8288151f243 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:32.934866 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf0328-ac60-4a6b-898d-45fae419b2b8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:32.951056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-7a37eebe-0067-4a2d-b2e4-b8288151f243 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:32.951056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:32.951056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:32.951056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:32.951056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:32.951056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:32.951056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:32.951056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:32.951056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:32.951056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:32.951056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:32.951056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:32.951056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:32.951056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:32.951056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:32.951056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:32.952043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:32.952043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:32.952043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:32.952043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:32.952043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:32.952043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:32.952043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:32.952043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:32.952043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:32.952043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:32.952043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:32.952043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:32.952043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:32.952043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:32.952043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:32.952043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:32.952043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:32.952926 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:32.952926 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:32.952926 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:32.955090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 309/598] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:32.956214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37eebe-0067-4a2d-b2e4-b8288151f243 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:32.982913 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41d271-2fba-469b-9ea9-18f117bd9ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:32.984142 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41d271-2fba-469b-9ea9-18f117bd9ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:32.984433 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41d271-2fba-469b-9ea9-18f117bd9ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:32.984707 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41d271-2fba-469b-9ea9-18f117bd9ae1 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:32.985355 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41d271-2fba-469b-9ea9-18f117bd9ae1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:32.985662 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41d271-2fba-469b-9ea9-18f117bd9ae1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:32.987660 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41d271-2fba-469b-9ea9-18f117bd9ae1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:33.005440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 290/599] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:33.006873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41d271-2fba-469b-9ea9-18f117bd9ae1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:33.065994 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:33.083135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37eebe-0067-4a2d-b2e4-b8288151f243 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:33.126514 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41d271-2fba-469b-9ea9-18f117bd9ae1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:33.133844 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18be51a-2cd4-4cd3-b448-e334ad8ad52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:33.134509 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18be51a-2cd4-4cd3-b448-e334ad8ad52f None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:33.134793 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18be51a-2cd4-4cd3-b448-e334ad8ad52f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:33.135069 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18be51a-2cd4-4cd3-b448-e334ad8ad52f None admin] PATH_INFO: `/v3/projects/8e201e5f281a4977be503e22daf149d7/users/eb3f6deffb54467f8b923956ed2a6acf/roles/a9bbb011afca4176a3def9356cb5f165` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:33.159689 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18be51a-2cd4-4cd3-b448-e334ad8ad52f None admin] RBAC: Authorizing `identity:create_grant(project_id=8e201e5f281a4977be503e22daf149d7, user_id=eb3f6deffb54467f8b923956ed2a6acf, role_id=a9bbb011afca4176a3def9356cb5f165)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:33.161472 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18be51a-2cd4-4cd3-b448-e334ad8ad52f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:33.168158 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58142309-a6d0-47be-9a33-d6e9a486a2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:33.168972 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58142309-a6d0-47be-9a33-d6e9a486a2df None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:33.169326 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58142309-a6d0-47be-9a33-d6e9a486a2df None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:33.169673 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58142309-a6d0-47be-9a33-d6e9a486a2df None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:33.170401 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58142309-a6d0-47be-9a33-d6e9a486a2df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:33.170775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58142309-a6d0-47be-9a33-d6e9a486a2df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:33.172314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58142309-a6d0-47be-9a33-d6e9a486a2df None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:33.188700 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 291/600] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:33.188700 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58142309-a6d0-47be-9a33-d6e9a486a2df None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:33.234303 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 310/601] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 12:23:32 2022] PUT /identity/v3/projects/8e201e5f281a4977be503e22daf149d7/users/eb3f6deffb54467f8b923956ed2a6acf/roles/a9bbb011afca4176a3def9356cb5f165 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:23:33.236025 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18be51a-2cd4-4cd3-b448-e334ad8ad52f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:33.295409 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:33.378278 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58142309-a6d0-47be-9a33-d6e9a486a2df None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:33.386030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:33.443923 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95200f2-7a40-43f9-83cb-9d4e1ce6522a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:33.444637 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95200f2-7a40-43f9-83cb-9d4e1ce6522a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:33.444924 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95200f2-7a40-43f9-83cb-9d4e1ce6522a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:33.445200 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95200f2-7a40-43f9-83cb-9d4e1ce6522a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:33.445592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18be51a-2cd4-4cd3-b448-e334ad8ad52f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:33.453759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95200f2-7a40-43f9-83cb-9d4e1ce6522a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:33.456033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95200f2-7a40-43f9-83cb-9d4e1ce6522a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:33.477001 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b95200f2-7a40-43f9-83cb-9d4e1ce6522a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:33.477001 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:33.477001 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:33.477001 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:33.477001 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:33.477001 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:33.477001 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:33.477001 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:33.477001 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:33.477001 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:33.477001 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:33.477001 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:33.477001 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:23:33.477001 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:23:33.477001 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:23:33.477931 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:23:33.477931 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:33.477931 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:33.477931 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:33.477931 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:33.477931 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:33.477931 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:33.477931 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:33.477931 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:33.477931 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:33.477931 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:33.477931 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:33.477931 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:33.477931 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:33.477931 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:33.477931 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:33.477931 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:33.478824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:23:33.478824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:23:33.478824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:23:33.478824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:23:33.478824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:23:33.478824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:23:33.478824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:33.478824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:33.482351 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 292/602] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:33.487879 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95200f2-7a40-43f9-83cb-9d4e1ce6522a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:33.511927 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4154393-cd91-4d88-a9ef-83bf365942c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:33.512596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4154393-cd91-4d88-a9ef-83bf365942c1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:33.512881 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4154393-cd91-4d88-a9ef-83bf365942c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:33.513156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4154393-cd91-4d88-a9ef-83bf365942c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:33.528508 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4154393-cd91-4d88-a9ef-83bf365942c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:33.529966 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4154393-cd91-4d88-a9ef-83bf365942c1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:33.551748 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-e4154393-cd91-4d88-a9ef-83bf365942c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:33.551748 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:33.551748 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:33.551748 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:33.551748 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:33.551748 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:33.551748 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:33.551748 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:33.551748 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:33.551748 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:33.551748 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:33.551748 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:33.551748 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:23:33.551748 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:23:33.551748 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:23:33.552704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:23:33.552704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:33.552704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:33.552704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:33.552704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:33.552704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:33.552704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:33.552704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:33.552704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:33.552704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:33.552704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:33.552704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:33.552704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:33.552704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:33.552704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:33.552704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:33.552704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:33.554818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:23:33.554818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:23:33.554818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:23:33.554818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:23:33.554818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:23:33.554818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:23:33.554818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:33.554818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:33.554818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 311/603] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:33.559010 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4154393-cd91-4d88-a9ef-83bf365942c1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:33.617183 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95200f2-7a40-43f9-83cb-9d4e1ce6522a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:33.654029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d77aac5-b160-4b77-b292-7924f3cf8861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:33.654974 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77aac5-b160-4b77-b292-7924f3cf8861 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:33.655496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77aac5-b160-4b77-b292-7924f3cf8861 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:33.655988 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77aac5-b160-4b77-b292-7924f3cf8861 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:33.656959 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d77aac5-b160-4b77-b292-7924f3cf8861 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:33.657463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d77aac5-b160-4b77-b292-7924f3cf8861 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:33.659118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d77aac5-b160-4b77-b292-7924f3cf8861 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:33.672125 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 293/604] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:23:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:33.684194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d77aac5-b160-4b77-b292-7924f3cf8861 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:33.726761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:33.739556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:33.776893 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4154393-cd91-4d88-a9ef-83bf365942c1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:33.832461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d12d3-c331-40cd-aadf-aa8a8e8bb2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:33.833076 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d12d3-c331-40cd-aadf-aa8a8e8bb2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:33.833357 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d12d3-c331-40cd-aadf-aa8a8e8bb2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:33.833631 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d12d3-c331-40cd-aadf-aa8a8e8bb2e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:33.834375 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d12d3-c331-40cd-aadf-aa8a8e8bb2e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:33.836258 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d12d3-c331-40cd-aadf-aa8a8e8bb2e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:33.839578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d12d3-c331-40cd-aadf-aa8a8e8bb2e5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:33.851447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d77aac5-b160-4b77-b292-7924f3cf8861 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:33.865425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 312/605] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:23:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:33.872046 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d12d3-c331-40cd-aadf-aa8a8e8bb2e5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:33.923108 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f25a3a-1184-4c30-b2d9-0c5bae250fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:33.924376 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f25a3a-1184-4c30-b2d9-0c5bae250fad None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:33.924664 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f25a3a-1184-4c30-b2d9-0c5bae250fad None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:33.924943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f25a3a-1184-4c30-b2d9-0c5bae250fad None admin] PATH_INFO: `/v3/projects/8e201e5f281a4977be503e22daf149d7/users/35dd27fd356241b885706375d880222f/roles/a9bbb011afca4176a3def9356cb5f165` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:33.960540 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f25a3a-1184-4c30-b2d9-0c5bae250fad None admin] RBAC: Authorizing `identity:create_grant(project_id=8e201e5f281a4977be503e22daf149d7, user_id=35dd27fd356241b885706375d880222f, role_id=a9bbb011afca4176a3def9356cb5f165)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:33.962015 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f25a3a-1184-4c30-b2d9-0c5bae250fad None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:34.006038 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d12d3-c331-40cd-aadf-aa8a8e8bb2e5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:34.041192 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ad4a9-40c8-4caf-bdd7-9be6ebe8940c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:34.041858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ad4a9-40c8-4caf-bdd7-9be6ebe8940c None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:34.042141 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ad4a9-40c8-4caf-bdd7-9be6ebe8940c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:34.042419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ad4a9-40c8-4caf-bdd7-9be6ebe8940c None admin] PATH_INFO: `/v3/projects/8e201e5f281a4977be503e22daf149d7/users/5db814952f9f466abf6148b7a0db2173/roles/a9bbb011afca4176a3def9356cb5f165` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:34.054820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:34.059297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ad4a9-40c8-4caf-bdd7-9be6ebe8940c None admin] RBAC: Authorizing `identity:create_grant(project_id=8e201e5f281a4977be503e22daf149d7, user_id=5db814952f9f466abf6148b7a0db2173, role_id=a9bbb011afca4176a3def9356cb5f165)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:34.060761 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ad4a9-40c8-4caf-bdd7-9be6ebe8940c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:34.087789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 294/606] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 12:23:33 2022] PUT /identity/v3/projects/8e201e5f281a4977be503e22daf149d7/users/35dd27fd356241b885706375d880222f/roles/a9bbb011afca4176a3def9356cb5f165 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:23:34.089597 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f379bd-566e-40b0-a25e-6df554b2b5b3 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:34.090118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f379bd-566e-40b0-a25e-6df554b2b5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:34.090622 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f379bd-566e-40b0-a25e-6df554b2b5b3 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:34.097078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 295/607] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:34.099404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e803078-399e-449d-9f90-1eb41b63d4f5 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:34.099925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e803078-399e-449d-9f90-1eb41b63d4f5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:34.100479 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e803078-399e-449d-9f90-1eb41b63d4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:34.123458 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8e803078-399e-449d-9f90-1eb41b63d4f5 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:34.125155 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8e803078-399e-449d-9f90-1eb41b63d4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:34.157696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 313/608] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 12:23:33 2022] PUT /identity/v3/projects/8e201e5f281a4977be503e22daf149d7/users/5db814952f9f466abf6148b7a0db2173/roles/a9bbb011afca4176a3def9356cb5f165 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:23:34.237172 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 296/609] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:34.291227 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ad4a9-40c8-4caf-bdd7-9be6ebe8940c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:34.410523 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ad4a9-40c8-4caf-bdd7-9be6ebe8940c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:34.450967 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ea147-753d-4e35-942b-7471f45c4c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:34.452079 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ea147-753d-4e35-942b-7471f45c4c0e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:34.452578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ea147-753d-4e35-942b-7471f45c4c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:34.453064 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ea147-753d-4e35-942b-7471f45c4c0e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:34.464424 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ea147-753d-4e35-942b-7471f45c4c0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:34.468618 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d56a7-ebc7-4526-919a-f858c054a5b7 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:34.469070 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d56a7-ebc7-4526-919a-f858c054a5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:34.469247 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d56a7-ebc7-4526-919a-f858c054a5b7 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:34.469792 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-db9ea147-753d-4e35-942b-7471f45c4c0e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:34.469792 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:34.469792 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:34.469792 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:34.469792 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:34.469792 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:34.469792 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:34.469792 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:34.469792 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:34.469792 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:34.469792 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:34.469792 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:34.469792 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:34.469792 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:34.469792 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:34.469792 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:34.470696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:34.470696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:34.470696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:34.470696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:34.470696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:34.470696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:34.470696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:34.470696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:34.470696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:34.470696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:34.470696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:34.470696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:34.470696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:34.470696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:34.470696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:34.470696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:34.471584 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:34.471584 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:34.471584 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:34.471584 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:34.471584 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:34.471584 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:34.471584 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:34.471584 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:34.471584 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:34.471584 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:34.471584 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:34.471584 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:34.472213 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 297/610] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:34.473704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 314/611] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 12:23:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:34.477847 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d56a7-ebc7-4526-919a-f858c054a5b7 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:34.480453 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ad52b-4ce7-4900-b537-3f637435bf58 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:34.480741 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ad52b-4ce7-4900-b537-3f637435bf58 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:34.481020 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ad52b-4ce7-4900-b537-3f637435bf58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:34.482006 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:34.502730 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-759ad52b-4ce7-4900-b537-3f637435bf58 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:34.504558 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-759ad52b-4ce7-4900-b537-3f637435bf58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:34.562287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:34.603660 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 315/612] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 10 12:23:34.643470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d56a7-ebc7-4526-919a-f858c054a5b7 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:34.653766 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ad52b-4ce7-4900-b537-3f637435bf58 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:34.712094 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeef2e4-3c27-4df4-8480-13ebbc191de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:34.713089 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeef2e4-3c27-4df4-8480-13ebbc191de2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:34.713586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeef2e4-3c27-4df4-8480-13ebbc191de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:34.714068 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeef2e4-3c27-4df4-8480-13ebbc191de2 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:34.714920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeef2e4-3c27-4df4-8480-13ebbc191de2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:34.715441 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeef2e4-3c27-4df4-8480-13ebbc191de2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:34.717100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeef2e4-3c27-4df4-8480-13ebbc191de2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:34.738965 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 298/613] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 12:23:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:34.741637 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeef2e4-3c27-4df4-8480-13ebbc191de2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:34.800364 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ad52b-4ce7-4900-b537-3f637435bf58 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:34.858819 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d6a7f-f597-4d8a-9957-178b22afeb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:34.860969 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d6a7f-f597-4d8a-9957-178b22afeb9c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:34.862732 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d6a7f-f597-4d8a-9957-178b22afeb9c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:34.863233 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d6a7f-f597-4d8a-9957-178b22afeb9c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:34.865289 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:34.893142 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3d6a7f-f597-4d8a-9957-178b22afeb9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:34.900878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-cb3d6a7f-f597-4d8a-9957-178b22afeb9c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:34.900878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:34.900878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:34.900878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:34.900878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:34.900878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:34.900878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:34.900878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:34.900878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:34.900878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:34.900878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:34.900878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:34.900878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:34.900878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:34.900878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:34.900878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:34.901853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:34.901853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:34.901853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:34.901853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:34.901853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:34.901853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:34.901853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:34.901853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:34.901853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:34.901853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:34.901853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:34.901853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:34.901853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:34.901853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:34.901853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:34.901853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:34.902687 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:34.902687 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:34.902687 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:34.902687 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:34.902687 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:34.902687 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:34.902687 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:34.902687 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:34.902687 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:34.902687 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:34.902687 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:34.902687 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:34.909114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 316/614] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 12:23:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:23:34.915115 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d6a7f-f597-4d8a-9957-178b22afeb9c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:34.949503 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeef2e4-3c27-4df4-8480-13ebbc191de2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:35.001532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:35.112804 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58506e-10f4-496f-8b9e-121591c1fc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:35.113892 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58506e-10f4-496f-8b9e-121591c1fc58 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:35.114440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58506e-10f4-496f-8b9e-121591c1fc58 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:35.114920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58506e-10f4-496f-8b9e-121591c1fc58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:35.134443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d6a7f-f597-4d8a-9957-178b22afeb9c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:35.137852 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58506e-10f4-496f-8b9e-121591c1fc58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:35.143947 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58506e-10f4-496f-8b9e-121591c1fc58 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:35.166360 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-0f58506e-10f4-496f-8b9e-121591c1fc58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:35.166360 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:35.166360 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:35.166360 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:35.166360 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:35.166360 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:35.166360 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:35.166360 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:35.166360 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:35.166360 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:35.166360 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:35.166360 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:35.166360 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:35.166360 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:35.166360 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:35.166360 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:35.167430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:35.167430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:35.167430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:35.167430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:35.167430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:35.167430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:35.167430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:35.167430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:35.167430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:35.167430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:35.167430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:35.167430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:35.167430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:35.167430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:35.167430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:35.167430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:35.167430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:35.168352 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:35.168352 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:35.168352 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:35.170643 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 299/615] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 429 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:35.175904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58506e-10f4-496f-8b9e-121591c1fc58 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:35.263899 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98d9b8-5e24-4cd1-a684-535cd73285ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:35.264537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98d9b8-5e24-4cd1-a684-535cd73285ed None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:35.264822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98d9b8-5e24-4cd1-a684-535cd73285ed None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:35.265099 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98d9b8-5e24-4cd1-a684-535cd73285ed None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:35.265771 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a98d9b8-5e24-4cd1-a684-535cd73285ed None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:35.266080 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a98d9b8-5e24-4cd1-a684-535cd73285ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:35.267479 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a98d9b8-5e24-4cd1-a684-535cd73285ed None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:35.290166 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58506e-10f4-496f-8b9e-121591c1fc58 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:35.297367 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 317/616] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 12:23:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:23:35.302840 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98d9b8-5e24-4cd1-a684-535cd73285ed None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:35.327623 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:35.332933 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ff3c6-38ac-407d-97b6-ab97ddd66185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:35.333789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ff3c6-38ac-407d-97b6-ab97ddd66185 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:35.336441 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ff3c6-38ac-407d-97b6-ab97ddd66185 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:35.336939 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ff3c6-38ac-407d-97b6-ab97ddd66185 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:35.337823 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9ff3c6-38ac-407d-97b6-ab97ddd66185 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:35.339536 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9ff3c6-38ac-407d-97b6-ab97ddd66185 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:35.341100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9ff3c6-38ac-407d-97b6-ab97ddd66185 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:35.365161 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 300/617] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:23:35.371335 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ff3c6-38ac-407d-97b6-ab97ddd66185 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:35.486999 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ff3c6-38ac-407d-97b6-ab97ddd66185 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:35.530481 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c2f97-5397-4989-aff0-acf8735c7182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:35.531682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c2f97-5397-4989-aff0-acf8735c7182 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:35.532192 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c2f97-5397-4989-aff0-acf8735c7182 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:35.532924 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c2f97-5397-4989-aff0-acf8735c7182 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:35.543337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c2f97-5397-4989-aff0-acf8735c7182 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:35.545557 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c2f97-5397-4989-aff0-acf8735c7182 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:35.549775 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98d9b8-5e24-4cd1-a684-535cd73285ed None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:35.551353 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:35.562418 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-1b5c2f97-5397-4989-aff0-acf8735c7182 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 12:23:35.562418 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:35.562418 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:35.562418 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:35.562418 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:35.562418 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:35.562418 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:35.562418 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:35.562418 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:35.562418 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:35.562418 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:35.562418 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:35.562418 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:23:35.562418 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:23:35.562418 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:23:35.562418 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:23:35.567337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:35.567337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:35.567337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:23:35.567337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:35.567337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:23:35.567337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:35.567337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:35.567337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:35.567337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:35.567337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:35.567337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:35.567337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:35.567337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:35.567337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:35.567337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:35.567337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:35.567337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:35.567337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:35.568395 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:35.568395 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:35.568395 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:23:35.568395 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:23:35.568395 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:23:35.568395 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:23:35.568395 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:23:35.568395 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:23:35.568395 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 12:23:35.568395 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:35.568395 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 301/618] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:23:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:35.568395 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80348833-f903-4557-9cff-b34d36e939b9 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:35.568395 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80348833-f903-4557-9cff-b34d36e939b9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:35.570366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80348833-f903-4557-9cff-b34d36e939b9 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:35.575803 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 302/619] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:35.575803 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80348833-f903-4557-9cff-b34d36e939b9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:35.599517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b45a1-52dc-47e5-82a0-68bfb9b77525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:35.600148 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b45a1-52dc-47e5-82a0-68bfb9b77525 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:35.600476 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b45a1-52dc-47e5-82a0-68bfb9b77525 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:35.600754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b45a1-52dc-47e5-82a0-68bfb9b77525 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:35.611896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b45a1-52dc-47e5-82a0-68bfb9b77525 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:35.613460 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b45a1-52dc-47e5-82a0-68bfb9b77525 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:35.621917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:35.632393 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-822b45a1-52dc-47e5-82a0-68bfb9b77525 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:35.632393 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:35.632393 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:35.632393 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:35.632393 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:35.632393 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:35.632393 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:35.632393 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:35.632393 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:35.632393 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:35.632393 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:35.632393 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:35.632393 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:35.632393 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:35.632393 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:35.632393 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:35.638855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:35.638855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:35.638855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:35.638855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:35.638855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:35.638855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:35.638855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:35.638855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:35.638855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:35.638855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:35.638855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:35.638855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:35.638855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:35.638855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:35.638855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:35.638855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:35.638855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:35.639814 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:35.639814 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:35.639814 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:35.639814 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 318/620] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:35.639814 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d7e08-b235-4b9f-994d-4a869a18d165 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:35.639814 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d7e08-b235-4b9f-994d-4a869a18d165 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:35.639814 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d7e08-b235-4b9f-994d-4a869a18d165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:35.661190 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-325d7e08-b235-4b9f-994d-4a869a18d165 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:35.662601 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-325d7e08-b235-4b9f-994d-4a869a18d165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:35.708585 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80348833-f903-4557-9cff-b34d36e939b9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:35.757818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 319/621] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:35.757818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d7e08-b235-4b9f-994d-4a869a18d165 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:35.758120 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2a664-359a-46e5-badf-3495b8b181e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:35.758450 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2a664-359a-46e5-badf-3495b8b181e5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:35.760082 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2a664-359a-46e5-badf-3495b8b181e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:35.760082 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2a664-359a-46e5-badf-3495b8b181e5 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:35.764314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa2a664-359a-46e5-badf-3495b8b181e5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:35.764689 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa2a664-359a-46e5-badf-3495b8b181e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:35.766134 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa2a664-359a-46e5-badf-3495b8b181e5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:35.798114 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 303/622] 10.222.0.49 () {56 vars in 1137 bytes} [Wed Aug 10 12:23:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:35.799847 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2a664-359a-46e5-badf-3495b8b181e5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:35.875998 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:35.896858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d7e08-b235-4b9f-994d-4a869a18d165 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:35.923889 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:35.931924 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2a664-359a-46e5-badf-3495b8b181e5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:35.937979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580bf08-c239-41c5-80c1-0388ce2e4060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:35.938652 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580bf08-c239-41c5-80c1-0388ce2e4060 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:35.938934 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580bf08-c239-41c5-80c1-0388ce2e4060 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:35.939220 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580bf08-c239-41c5-80c1-0388ce2e4060 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:35.939857 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580bf08-c239-41c5-80c1-0388ce2e4060 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:35.940156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580bf08-c239-41c5-80c1-0388ce2e4060 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:35.941570 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580bf08-c239-41c5-80c1-0388ce2e4060 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:35.954587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 320/623] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:35.955709 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580bf08-c239-41c5-80c1-0388ce2e4060 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:35.970020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb01f9-cfe9-4320-954e-a9ccbba50454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:35.970931 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb01f9-cfe9-4320-954e-a9ccbba50454 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:35.972441 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb01f9-cfe9-4320-954e-a9ccbba50454 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:35.972441 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb01f9-cfe9-4320-954e-a9ccbba50454 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:35.984445 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb01f9-cfe9-4320-954e-a9ccbba50454 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:35.989978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-87cb01f9-cfe9-4320-954e-a9ccbba50454 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:35.989978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:35.989978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:35.989978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:35.989978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:35.989978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:35.989978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:35.989978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:35.989978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:35.989978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:35.989978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:35.989978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:35.989978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:35.989978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:35.989978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:35.989978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:35.990967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:35.990967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:35.990967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:35.990967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:35.990967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:35.990967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:35.990967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:35.990967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:35.990967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:35.990967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:35.990967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:35.990967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:35.990967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:35.990967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:35.990967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:35.990967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:35.991867 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:35.991867 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:35.991867 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:35.991867 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:35.991867 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:35.991867 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:35.991867 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:35.991867 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:35.991867 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:35.991867 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:35.991867 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:35.991867 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:35.994411 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 304/624] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 12:23:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:35.995442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb01f9-cfe9-4320-954e-a9ccbba50454 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:36.068609 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580bf08-c239-41c5-80c1-0388ce2e4060 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:36.108803 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce684c4-8f7d-4b5b-a7fe-f5cca0f8b130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:36.109588 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce684c4-8f7d-4b5b-a7fe-f5cca0f8b130 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:36.109841 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce684c4-8f7d-4b5b-a7fe-f5cca0f8b130 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:36.110188 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce684c4-8f7d-4b5b-a7fe-f5cca0f8b130 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:36.115424 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb01f9-cfe9-4320-954e-a9ccbba50454 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:36.126832 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:36.156784 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c623cd56-3559-4390-ad7f-5288d21bd92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:36.158087 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623cd56-3559-4390-ad7f-5288d21bd92c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:36.158554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623cd56-3559-4390-ad7f-5288d21bd92c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:36.159012 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623cd56-3559-4390-ad7f-5288d21bd92c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:36.173670 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c623cd56-3559-4390-ad7f-5288d21bd92c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:36.175116 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c623cd56-3559-4390-ad7f-5288d21bd92c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:36.189537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-c623cd56-3559-4390-ad7f-5288d21bd92c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 12:23:36.189537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:36.189537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:36.189537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:36.189537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:36.189537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:36.189537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:36.189537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:36.189537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:36.189537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:36.189537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:36.189537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:36.189537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:23:36.189537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:23:36.189537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:23:36.189537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:23:36.190566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:36.190566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:36.190566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:23:36.190566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:36.190566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:23:36.190566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:36.190566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:36.190566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:36.190566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:36.190566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:36.190566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:36.190566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:36.190566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:36.190566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:36.190566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:36.190566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:36.190566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:36.190566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:36.191584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:36.191584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:36.191584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:23:36.191584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:23:36.191584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:23:36.191584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:23:36.191584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:23:36.191584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:23:36.191584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 12:23:36.191584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:36.191584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 305/625] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:23:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:36.192180 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c623cd56-3559-4390-ad7f-5288d21bd92c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:36.202558 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce684c4-8f7d-4b5b-a7fe-f5cca0f8b130 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:36.204889 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce684c4-8f7d-4b5b-a7fe-f5cca0f8b130 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:36.209194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:36.316883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c623cd56-3559-4390-ad7f-5288d21bd92c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:36.332271 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 321/626] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:23:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:23:36.333701 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce684c4-8f7d-4b5b-a7fe-f5cca0f8b130 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:36.357416 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a900c1-879a-4799-b8b1-a239ba8dcf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:36.358021 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a900c1-879a-4799-b8b1-a239ba8dcf0a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:36.358297 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a900c1-879a-4799-b8b1-a239ba8dcf0a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:36.358565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a900c1-879a-4799-b8b1-a239ba8dcf0a None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:36.359243 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a900c1-879a-4799-b8b1-a239ba8dcf0a None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:36.359549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a900c1-879a-4799-b8b1-a239ba8dcf0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:36.360901 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a900c1-879a-4799-b8b1-a239ba8dcf0a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:36.370011 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 306/627] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 12:23:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:36.370929 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a900c1-879a-4799-b8b1-a239ba8dcf0a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:36.396646 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:36.460381 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:36.471417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce684c4-8f7d-4b5b-a7fe-f5cca0f8b130 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:36.485166 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a900c1-879a-4799-b8b1-a239ba8dcf0a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:36.517435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26804609-a7c5-4f7c-82a9-ba0dfb381463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:36.518424 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26804609-a7c5-4f7c-82a9-ba0dfb381463 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:36.518700 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26804609-a7c5-4f7c-82a9-ba0dfb381463 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:36.519110 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26804609-a7c5-4f7c-82a9-ba0dfb381463 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:36.519964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26804609-a7c5-4f7c-82a9-ba0dfb381463 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:36.520264 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26804609-a7c5-4f7c-82a9-ba0dfb381463 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:36.521679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26804609-a7c5-4f7c-82a9-ba0dfb381463 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:36.524055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e99667-40dc-4356-ac8d-9af6caa5bff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:36.524685 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e99667-40dc-4356-ac8d-9af6caa5bff9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:36.524963 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e99667-40dc-4356-ac8d-9af6caa5bff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:36.525239 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e99667-40dc-4356-ac8d-9af6caa5bff9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:36.541973 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e99667-40dc-4356-ac8d-9af6caa5bff9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:36.541973 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e99667-40dc-4356-ac8d-9af6caa5bff9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:36.545007 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 322/628] 10.222.0.49 () {56 vars in 1135 bytes} [Wed Aug 10 12:23:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:36.545990 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26804609-a7c5-4f7c-82a9-ba0dfb381463 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:36.558365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-67e99667-40dc-4356-ac8d-9af6caa5bff9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:36.558365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:36.558365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:36.558365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:36.558365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:36.558365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:36.558365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:36.558365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:36.558365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:36.558365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:36.558365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:36.558365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:36.558365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:36.558365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:36.558365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:36.558365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:36.559692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:36.559692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:36.559692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:36.559692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:36.559692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:36.559692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:36.559692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:36.559692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:36.559692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:36.559692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:36.559692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:36.559692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:36.559692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:36.559692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:36.559692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:36.559692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:36.559692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:36.561140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:36.561140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:36.561140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:36.561140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 307/629] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:36.561140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d11eb-5179-4f6a-a85c-d840b2091a2b None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:36.561140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d11eb-5179-4f6a-a85c-d840b2091a2b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:36.561561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d11eb-5179-4f6a-a85c-d840b2091a2b None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:36.562744 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 308/630] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:36.567396 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d11eb-5179-4f6a-a85c-d840b2091a2b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:36.648780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26804609-a7c5-4f7c-82a9-ba0dfb381463 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:36.650231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:36.668361 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d11eb-5179-4f6a-a85c-d840b2091a2b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:36.681944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b14155-1382-45b5-a4f1-04bb3079e2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:36.682637 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b14155-1382-45b5-a4f1-04bb3079e2f1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:36.682918 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b14155-1382-45b5-a4f1-04bb3079e2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:36.683218 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b14155-1382-45b5-a4f1-04bb3079e2f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:36.693539 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b14155-1382-45b5-a4f1-04bb3079e2f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:36.694985 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:36.695580 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b14155-1382-45b5-a4f1-04bb3079e2f1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:36.702714 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7391c-f431-45ba-b0e5-5ddb9a7737ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:36.703356 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7391c-f431-45ba-b0e5-5ddb9a7737ef None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:36.703632 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7391c-f431-45ba-b0e5-5ddb9a7737ef None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:36.706103 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7391c-f431-45ba-b0e5-5ddb9a7737ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:36.710300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-27b14155-1382-45b5-a4f1-04bb3079e2f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:36.710300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:36.710300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:36.710300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:36.710300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:36.710300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:36.710300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:36.710300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:36.710300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:36.710300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:36.710300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:36.710300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:36.710300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:36.710300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:36.710300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:36.710300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:36.711497 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:36.711497 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:36.711497 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:36.711497 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:36.711497 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:36.711497 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:36.711497 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:36.711497 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:36.711497 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:36.711497 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:36.711497 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:36.711497 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:36.711497 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:36.711497 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:36.711497 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:36.711497 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:36.711497 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:36.712659 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:36.712659 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:36.712659 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:36.712659 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 323/631] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:36.712884 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18e423-cbf9-4fee-8633-31006e88e41a None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:36.713167 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18e423-cbf9-4fee-8633-31006e88e41a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:36.713443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18e423-cbf9-4fee-8633-31006e88e41a None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:36.714669 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 324/632] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:36.715482 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f18e423-cbf9-4fee-8633-31006e88e41a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:36.719683 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7391c-f431-45ba-b0e5-5ddb9a7737ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:36.721890 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7391c-f431-45ba-b0e5-5ddb9a7737ef None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:36.739524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d0b7391c-f431-45ba-b0e5-5ddb9a7737ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:36.739524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:36.739524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:36.739524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:36.739524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:36.739524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:36.739524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:36.739524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:36.739524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:36.739524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:36.739524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:36.739524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:36.739524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:36.739524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:36.739524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:36.739524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:36.740755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:36.740755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:36.740755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:36.740755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:36.740755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:36.740755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:36.740755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:36.740755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:36.740755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:36.740755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:36.740755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:36.740755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:36.740755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:36.740755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:36.740755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:36.740755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:36.740755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:36.741899 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:36.741899 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:36.741899 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:36.743353 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 309/633] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:36.744688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302eb50-d3a6-4fb8-ade4-bcfb078556ea None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:36.744973 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302eb50-d3a6-4fb8-ade4-bcfb078556ea None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:36.745249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302eb50-d3a6-4fb8-ade4-bcfb078556ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:36.763664 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5302eb50-d3a6-4fb8-ade4-bcfb078556ea None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:36.764853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5302eb50-d3a6-4fb8-ade4-bcfb078556ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:36.820462 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f18e423-cbf9-4fee-8633-31006e88e41a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:36.830307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 310/634] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:36.830958 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302eb50-d3a6-4fb8-ade4-bcfb078556ea None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:36.854957 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34953d-5c11-428e-a018-5a33069ec6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:36.855592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34953d-5c11-428e-a018-5a33069ec6f7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:36.855872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34953d-5c11-428e-a018-5a33069ec6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:36.856146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34953d-5c11-428e-a018-5a33069ec6f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:36.856776 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34953d-5c11-428e-a018-5a33069ec6f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:36.857079 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34953d-5c11-428e-a018-5a33069ec6f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:36.858502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34953d-5c11-428e-a018-5a33069ec6f7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:36.871465 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 325/635] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:36.872627 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56256393-25c9-4650-ac50-87433bcf59d2 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:36.872907 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56256393-25c9-4650-ac50-87433bcf59d2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:36.873176 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56256393-25c9-4650-ac50-87433bcf59d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:36.891126 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-56256393-25c9-4650-ac50-87433bcf59d2 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:36.892344 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-56256393-25c9-4650-ac50-87433bcf59d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:36.901872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:36.949593 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302eb50-d3a6-4fb8-ade4-bcfb078556ea None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:36.957316 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 326/636] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:36.958200 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56256393-25c9-4650-ac50-87433bcf59d2 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:36.962138 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:36.986101 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6495b78-ab04-4dc8-ba29-5a4310664358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:36.986701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6495b78-ab04-4dc8-ba29-5a4310664358 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:36.986976 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6495b78-ab04-4dc8-ba29-5a4310664358 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:36.987273 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6495b78-ab04-4dc8-ba29-5a4310664358 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:36.987957 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6495b78-ab04-4dc8-ba29-5a4310664358 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:36.988599 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6495b78-ab04-4dc8-ba29-5a4310664358 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:36.990415 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6495b78-ab04-4dc8-ba29-5a4310664358 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:37.001873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 311/637] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:37.002656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6495b78-ab04-4dc8-ba29-5a4310664358 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:37.063393 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56256393-25c9-4650-ac50-87433bcf59d2 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:37.097464 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604dad57-fa8d-4566-8b47-fbe40a7686f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:37.098148 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604dad57-fa8d-4566-8b47-fbe40a7686f4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:37.098423 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604dad57-fa8d-4566-8b47-fbe40a7686f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:37.098694 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604dad57-fa8d-4566-8b47-fbe40a7686f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:37.099358 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604dad57-fa8d-4566-8b47-fbe40a7686f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:37.099666 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604dad57-fa8d-4566-8b47-fbe40a7686f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:37.101150 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604dad57-fa8d-4566-8b47-fbe40a7686f4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:37.105170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6495b78-ab04-4dc8-ba29-5a4310664358 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:37.112395 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 327/638] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:37.113359 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604dad57-fa8d-4566-8b47-fbe40a7686f4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:37.120367 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:37.141920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5630496-d6b1-4481-8f85-073d31af2369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:37.142639 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5630496-d6b1-4481-8f85-073d31af2369 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:37.142992 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5630496-d6b1-4481-8f85-073d31af2369 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:37.143428 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5630496-d6b1-4481-8f85-073d31af2369 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:37.153830 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5630496-d6b1-4481-8f85-073d31af2369 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:37.158423 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-c5630496-d6b1-4481-8f85-073d31af2369 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:37.158423 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:37.158423 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:37.158423 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:37.158423 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:37.158423 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:37.158423 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:37.158423 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:37.158423 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:37.158423 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:37.158423 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:37.158423 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:37.158423 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:37.158423 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:37.158423 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:37.158423 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:37.159573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:37.159573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:37.159573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:37.159573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:37.159573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:37.159573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:37.159573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:37.159573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:37.159573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:37.159573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:37.159573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:37.159573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:37.159573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:37.159573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:37.159573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:37.159573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:37.160636 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:37.160636 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:37.160636 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:37.160636 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:37.160636 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:37.160636 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:37.160636 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:37.160636 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:37.160636 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:37.160636 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:37.160636 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:37.160636 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:37.163484 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 312/639] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 12:23:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:37.164322 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5630496-d6b1-4481-8f85-073d31af2369 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:37.193598 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:37.214934 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604dad57-fa8d-4566-8b47-fbe40a7686f4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:37.250093 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71613695-24b5-4fcd-96de-44808fc2f366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:37.250725 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71613695-24b5-4fcd-96de-44808fc2f366 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:37.251008 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71613695-24b5-4fcd-96de-44808fc2f366 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:37.251328 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71613695-24b5-4fcd-96de-44808fc2f366 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:37.258989 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71613695-24b5-4fcd-96de-44808fc2f366 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:37.260463 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71613695-24b5-4fcd-96de-44808fc2f366 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:37.264389 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5630496-d6b1-4481-8f85-073d31af2369 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:37.271758 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-71613695-24b5-4fcd-96de-44808fc2f366 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:37.271758 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:37.271758 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:37.271758 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:37.271758 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:37.271758 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:37.271758 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:37.271758 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:37.271758 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:37.271758 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:37.271758 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:37.271758 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:37.271758 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:23:37.271758 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:23:37.271758 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:23:37.272896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:23:37.272896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:37.272896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:37.272896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:37.272896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:37.272896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:37.272896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:37.272896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:37.272896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:37.272896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:37.272896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:37.272896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:37.272896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:37.272896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:37.272896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:37.272896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:37.272896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:37.278909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:23:37.278909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:23:37.278909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:23:37.278909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:23:37.278909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:23:37.278909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:23:37.278909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:37.278909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:37.278909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 328/640] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:37.278909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71613695-24b5-4fcd-96de-44808fc2f366 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:37.300713 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca47ea8-031e-458f-ba2b-2749f03a8365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:37.301346 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca47ea8-031e-458f-ba2b-2749f03a8365 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:37.301626 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca47ea8-031e-458f-ba2b-2749f03a8365 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:37.301902 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca47ea8-031e-458f-ba2b-2749f03a8365 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:37.312549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca47ea8-031e-458f-ba2b-2749f03a8365 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:37.317203 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-4ca47ea8-031e-458f-ba2b-2749f03a8365 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:37.317203 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:37.317203 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:37.317203 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:37.317203 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:37.317203 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:37.317203 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:37.317203 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:37.317203 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:37.317203 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:37.317203 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:37.317203 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:37.317203 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:37.317203 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:37.317203 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:37.317203 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:37.318380 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:37.318380 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:37.318380 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:37.318380 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:37.318380 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:37.318380 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:37.318380 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:37.318380 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:37.318380 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:37.318380 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:37.318380 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:37.318380 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:37.318380 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:37.318380 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:37.318380 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:37.318380 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:37.320211 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:37.320211 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:37.320211 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:37.320211 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:37.320211 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:37.320211 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:37.320211 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:37.320211 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:37.320211 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:37.320211 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:37.320211 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:37.320211 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:37.320211 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 313/641] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 12:23:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:37.320211 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca47ea8-031e-458f-ba2b-2749f03a8365 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:37.339748 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:37.376431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71613695-24b5-4fcd-96de-44808fc2f366 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:37.409998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:37.410337 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4996126-cdb1-4ea0-b9f8-ce817604576d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:37.410976 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4996126-cdb1-4ea0-b9f8-ce817604576d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:37.411282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4996126-cdb1-4ea0-b9f8-ce817604576d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:37.411555 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4996126-cdb1-4ea0-b9f8-ce817604576d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:37.419120 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4996126-cdb1-4ea0-b9f8-ce817604576d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:37.419512 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca47ea8-031e-458f-ba2b-2749f03a8365 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:37.421980 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4996126-cdb1-4ea0-b9f8-ce817604576d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:37.432260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-b4996126-cdb1-4ea0-b9f8-ce817604576d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:23:37.432260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:37.432260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:37.432260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:37.432260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:37.432260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:37.432260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:37.432260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:37.432260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:37.432260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:37.432260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:37.432260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:37.432260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:23:37.432260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:23:37.432260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:23:37.432260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:23:37.433434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:37.433434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:37.433434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:23:37.433434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:37.433434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:23:37.433434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:37.433434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:37.433434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:37.433434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:37.433434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:37.433434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:37.433434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:37.433434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:37.433434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:37.433434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:37.433434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:37.433434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:37.433434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:37.434754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:37.434754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:37.434754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:23:37.434754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:23:37.434754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:23:37.434754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:23:37.434754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:23:37.434754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:23:37.434754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:23:37.434754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:37.434754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 329/642] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Aug 10 12:23:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:37.434754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4996126-cdb1-4ea0-b9f8-ce817604576d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:37.457333 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b72e8-7961-437e-be8f-276006bc36cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:37.457959 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b72e8-7961-437e-be8f-276006bc36cf None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:37.458240 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b72e8-7961-437e-be8f-276006bc36cf None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:37.458509 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b72e8-7961-437e-be8f-276006bc36cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:37.465635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97b72e8-7961-437e-be8f-276006bc36cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:37.467118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97b72e8-7961-437e-be8f-276006bc36cf None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:37.478642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-e97b72e8-7961-437e-be8f-276006bc36cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:37.478642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:37.478642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:37.478642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:37.478642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:37.478642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:37.478642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:37.478642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:37.478642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:37.478642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:37.478642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:37.478642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:37.478642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:23:37.478642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:23:37.478642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:23:37.479775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:23:37.479775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:37.479775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:37.479775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:37.479775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:37.479775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:37.479775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:37.479775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:37.479775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:37.479775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:37.479775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:37.479775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:37.479775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:37.479775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:37.479775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:37.479775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:37.479775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:37.481014 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:23:37.481014 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:23:37.481014 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:23:37.481014 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:23:37.481014 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:23:37.481014 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:23:37.481014 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:37.481014 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:37.481014 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 314/643] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:37.481014 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b72e8-7961-437e-be8f-276006bc36cf None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:37.536997 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4996126-cdb1-4ea0-b9f8-ce817604576d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:37.559802 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:37.570730 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb3264-39df-4ff5-9f73-2407f49e8468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:37.571370 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb3264-39df-4ff5-9f73-2407f49e8468 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:37.571648 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb3264-39df-4ff5-9f73-2407f49e8468 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:37.571917 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb3264-39df-4ff5-9f73-2407f49e8468 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:37.577967 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb3264-39df-4ff5-9f73-2407f49e8468 None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:37.577967 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb3264-39df-4ff5-9f73-2407f49e8468 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:37.577967 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb3264-39df-4ff5-9f73-2407f49e8468 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:37.579475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b72e8-7961-437e-be8f-276006bc36cf None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:37.581791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 330/644] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 12:23:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:37.582630 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb3264-39df-4ff5-9f73-2407f49e8468 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:37.615707 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c17ba-5b59-422c-a173-da6b9831123b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:37.616407 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c17ba-5b59-422c-a173-da6b9831123b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:37.616686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c17ba-5b59-422c-a173-da6b9831123b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:37.616959 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c17ba-5b59-422c-a173-da6b9831123b None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:37.617643 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5c17ba-5b59-422c-a173-da6b9831123b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:37.617949 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5c17ba-5b59-422c-a173-da6b9831123b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:37.619435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5c17ba-5b59-422c-a173-da6b9831123b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:37.632758 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 315/645] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:23:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:37.633565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c17ba-5b59-422c-a173-da6b9831123b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:37.645460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:37.684502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb3264-39df-4ff5-9f73-2407f49e8468 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:37.717446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3b04a-6d09-46ef-9b79-e7056e673233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:37.718050 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3b04a-6d09-46ef-9b79-e7056e673233 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:37.718331 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3b04a-6d09-46ef-9b79-e7056e673233 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:37.718634 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3b04a-6d09-46ef-9b79-e7056e673233 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:37.719314 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3b04a-6d09-46ef-9b79-e7056e673233 None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:37.719621 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3b04a-6d09-46ef-9b79-e7056e673233 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:37.720965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3b04a-6d09-46ef-9b79-e7056e673233 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:37.729301 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 331/646] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 12:23:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:37.730120 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3b04a-6d09-46ef-9b79-e7056e673233 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:37.733191 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c17ba-5b59-422c-a173-da6b9831123b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:37.768399 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1938bc-e2a2-40c2-b6d5-c4c92b7611d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:37.769024 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1938bc-e2a2-40c2-b6d5-c4c92b7611d5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:37.769326 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1938bc-e2a2-40c2-b6d5-c4c92b7611d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:37.769582 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1938bc-e2a2-40c2-b6d5-c4c92b7611d5 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:37.770295 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1938bc-e2a2-40c2-b6d5-c4c92b7611d5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:37.770601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1938bc-e2a2-40c2-b6d5-c4c92b7611d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:37.772096 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1938bc-e2a2-40c2-b6d5-c4c92b7611d5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:37.782084 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:37.789590 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 316/647] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Aug 10 12:23:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:37.795634 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1938bc-e2a2-40c2-b6d5-c4c92b7611d5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:37.832944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3b04a-6d09-46ef-9b79-e7056e673233 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:37.856594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:37.867140 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6f0c4-c20f-452c-bace-ff2ff9dd7d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:37.867772 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6f0c4-c20f-452c-bace-ff2ff9dd7d46 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:37.869571 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6f0c4-c20f-452c-bace-ff2ff9dd7d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:37.869571 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6f0c4-c20f-452c-bace-ff2ff9dd7d46 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:37.869571 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa6f0c4-c20f-452c-bace-ff2ff9dd7d46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:37.869571 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa6f0c4-c20f-452c-bace-ff2ff9dd7d46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:37.870910 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa6f0c4-c20f-452c-bace-ff2ff9dd7d46 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:37.883212 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 332/648] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:23:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:37.884001 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6f0c4-c20f-452c-bace-ff2ff9dd7d46 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:37.889969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1938bc-e2a2-40c2-b6d5-c4c92b7611d5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:37.925591 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de9eec-2ced-4c6f-afc7-0c35429ec3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:37.926227 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de9eec-2ced-4c6f-afc7-0c35429ec3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:37.926504 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de9eec-2ced-4c6f-afc7-0c35429ec3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:37.926781 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de9eec-2ced-4c6f-afc7-0c35429ec3b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:37.937725 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de9eec-2ced-4c6f-afc7-0c35429ec3b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:37.939096 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de9eec-2ced-4c6f-afc7-0c35429ec3b8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:37.952873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-44de9eec-2ced-4c6f-afc7-0c35429ec3b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:37.952873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:37.952873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:37.952873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:37.952873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:37.952873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:37.952873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:37.952873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:37.952873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:37.952873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:37.952873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:37.952873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:37.952873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:37.952873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:37.952873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:37.952873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:37.953938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:37.953938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:37.953938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:37.953938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:37.953938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:37.953938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:37.953938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:37.953938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:37.953938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:37.953938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:37.953938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:37.953938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:37.953938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:37.953938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:37.953938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:37.953938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:37.953938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:37.954996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:37.954996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:37.954996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:37.954996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 317/649] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:37.955273 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de9eec-2ced-4c6f-afc7-0c35429ec3b8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:37.989893 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6f0c4-c20f-452c-bace-ff2ff9dd7d46 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:37.999710 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:38.024072 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd17d6a-cc46-43c7-b250-b449a3e163d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:38.024795 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd17d6a-cc46-43c7-b250-b449a3e163d0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:38.025162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd17d6a-cc46-43c7-b250-b449a3e163d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:38.025442 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd17d6a-cc46-43c7-b250-b449a3e163d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:38.026159 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd17d6a-cc46-43c7-b250-b449a3e163d0 None admin] RBAC: Adding query filter params (role.id=a9bbb011afca4176a3def9356cb5f165, scope.project.id=8e201e5f281a4977be503e22daf149d7, user.id=ab5a0d9c39d84cc7a91f5a752dcf2b70) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:38.026473 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd17d6a-cc46-43c7-b250-b449a3e163d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:38.027918 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd17d6a-cc46-43c7-b250-b449a3e163d0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:38.033884 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 333/650] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:23:37 2022] GET /identity/v3/role_assignments?user.id=ab5a0d9c39d84cc7a91f5a752dcf2b70&scope.project.id=8e201e5f281a4977be503e22daf149d7&role.id=a9bbb011afca4176a3def9356cb5f165 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:38.034780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd17d6a-cc46-43c7-b250-b449a3e163d0 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:38.063489 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de9eec-2ced-4c6f-afc7-0c35429ec3b8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:38.088270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:38.104191 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f71fd-0839-40b7-b92b-fe9153359cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:38.104939 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f71fd-0839-40b7-b92b-fe9153359cab None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:38.105298 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f71fd-0839-40b7-b92b-fe9153359cab None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:38.105703 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f71fd-0839-40b7-b92b-fe9153359cab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:38.118473 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4f71fd-0839-40b7-b92b-fe9153359cab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:38.120171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4f71fd-0839-40b7-b92b-fe9153359cab None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:38.137693 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-bc4f71fd-0839-40b7-b92b-fe9153359cab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:38.137693 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:38.137693 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:38.137693 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:38.137693 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:38.137693 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:38.137693 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:38.137693 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:38.137693 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:38.137693 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:38.137693 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:38.137693 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:38.137693 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:38.137693 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:38.137693 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:38.137693 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:38.138909 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:38.138909 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:38.138909 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:38.138909 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:38.138909 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:38.138909 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:38.138909 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:38.138909 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:38.138909 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:38.138909 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:38.138909 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:38.138909 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:38.138909 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:38.138909 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:38.138909 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:38.138909 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:38.138909 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:38.144705 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:38.144705 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:38.144705 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:38.144705 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 318/651] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:38.144705 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f71fd-0839-40b7-b92b-fe9153359cab None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:38.157997 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd17d6a-cc46-43c7-b250-b449a3e163d0 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:38.202806 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815112bf-d376-41d5-9247-4c00980050bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:38.203466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815112bf-d376-41d5-9247-4c00980050bd None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:38.203773 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815112bf-d376-41d5-9247-4c00980050bd None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:38.204019 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815112bf-d376-41d5-9247-4c00980050bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:38.253020 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:38.255009 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f71fd-0839-40b7-b92b-fe9153359cab None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:38.287757 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815112bf-d376-41d5-9247-4c00980050bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:38.289233 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815112bf-d376-41d5-9247-4c00980050bd None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:38.294924 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d94af-2b8d-40ec-bbd8-c71ce28e5c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:38.295625 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d94af-2b8d-40ec-bbd8-c71ce28e5c43 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:38.295903 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d94af-2b8d-40ec-bbd8-c71ce28e5c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:38.296176 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d94af-2b8d-40ec-bbd8-c71ce28e5c43 None admin] PATH_INFO: `/v3/projects/8e201e5f281a4977be503e22daf149d7/users/59395bfa505c4540b3957deee78eb698/roles/a9bbb011afca4176a3def9356cb5f165` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:38.311611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d94af-2b8d-40ec-bbd8-c71ce28e5c43 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e201e5f281a4977be503e22daf149d7, user_id=59395bfa505c4540b3957deee78eb698, role_id=a9bbb011afca4176a3def9356cb5f165)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:38.313048 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d94af-2b8d-40ec-bbd8-c71ce28e5c43 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:38.383925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 319/652] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 12:23:38 2022] PUT /identity/v3/projects/8e201e5f281a4977be503e22daf149d7/users/59395bfa505c4540b3957deee78eb698/roles/a9bbb011afca4176a3def9356cb5f165 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:23:38.385181 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d94af-2b8d-40ec-bbd8-c71ce28e5c43 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:38.404697 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 334/653] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:23:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:23:38.404697 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815112bf-d376-41d5-9247-4c00980050bd None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:38.428782 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:38.494930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:38.531154 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815112bf-d376-41d5-9247-4c00980050bd None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:38.535849 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d94af-2b8d-40ec-bbd8-c71ce28e5c43 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:38.575926 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff560106-9139-4f6d-b350-6d5dc90a2e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:38.576538 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff560106-9139-4f6d-b350-6d5dc90a2e37 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:38.576815 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff560106-9139-4f6d-b350-6d5dc90a2e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:38.577088 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff560106-9139-4f6d-b350-6d5dc90a2e37 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:38.577722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff560106-9139-4f6d-b350-6d5dc90a2e37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:38.578029 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff560106-9139-4f6d-b350-6d5dc90a2e37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:38.578553 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cde1fb-e1f1-49ee-a66e-03f7b41f0b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:38.579167 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cde1fb-e1f1-49ee-a66e-03f7b41f0b06 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:38.579428 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff560106-9139-4f6d-b350-6d5dc90a2e37 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:38.579540 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cde1fb-e1f1-49ee-a66e-03f7b41f0b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:38.579933 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cde1fb-e1f1-49ee-a66e-03f7b41f0b06 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:38.580363 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cde1fb-e1f1-49ee-a66e-03f7b41f0b06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:38.580667 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cde1fb-e1f1-49ee-a66e-03f7b41f0b06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:38.582011 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cde1fb-e1f1-49ee-a66e-03f7b41f0b06 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:38.592772 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 335/654] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:38.593694 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff560106-9139-4f6d-b350-6d5dc90a2e37 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:38.595133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 320/655] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:38.598377 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cde1fb-e1f1-49ee-a66e-03f7b41f0b06 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:38.688527 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:38.706992 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cde1fb-e1f1-49ee-a66e-03f7b41f0b06 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:38.721836 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff560106-9139-4f6d-b350-6d5dc90a2e37 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:38.745016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b290f-ee47-4647-b886-c71672e3cabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:38.746288 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b290f-ee47-4647-b886-c71672e3cabe None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:38.746796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b290f-ee47-4647-b886-c71672e3cabe None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:38.747308 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b290f-ee47-4647-b886-c71672e3cabe None admin] PATH_INFO: `/v3/users/glance` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:38.754968 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b290f-ee47-4647-b886-c71672e3cabe None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:38.755899 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:38.757071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b290f-ee47-4647-b886-c71672e3cabe None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:38.762092 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df28f5d-b4df-4442-afd2-c2fa21330b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:38.762836 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df28f5d-b4df-4442-afd2-c2fa21330b82 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:38.767334 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df28f5d-b4df-4442-afd2-c2fa21330b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:38.767334 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df28f5d-b4df-4442-afd2-c2fa21330b82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:38.771669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b09b290f-ee47-4647-b886-c71672e3cabe None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 12:23:38.771669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:38.771669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:38.771669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:38.771669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:38.771669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:38.771669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:38.771669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:38.771669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:38.771669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:38.771669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:38.771669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:38.771669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:23:38.771669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:23:38.771669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:23:38.771669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:23:38.773635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:38.773635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:38.773635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:23:38.773635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:38.773635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:23:38.773635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:38.773635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:38.773635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:38.773635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:38.773635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:38.773635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:38.773635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:38.773635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:38.773635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:38.773635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:38.773635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:38.773635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:38.773635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:38.774584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:38.774584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:38.774584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:23:38.774584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:23:38.774584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:23:38.774584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:23:38.774584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:23:38.774584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:23:38.774584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 12:23:38.774584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:38.779768 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df28f5d-b4df-4442-afd2-c2fa21330b82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:38.779768 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df28f5d-b4df-4442-afd2-c2fa21330b82 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:38.786154 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 321/656] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:23:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:38.786154 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b290f-ee47-4647-b886-c71672e3cabe None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:38.809617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-6df28f5d-b4df-4442-afd2-c2fa21330b82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:38.809617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:38.809617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:38.809617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:38.809617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:38.809617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:38.809617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:38.809617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:38.809617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:38.809617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:38.809617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:38.809617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:38.809617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:38.809617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:38.809617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:38.809617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:38.811156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:38.811156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:38.811156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:38.811156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:38.811156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:38.811156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:38.811156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:38.811156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:38.811156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:38.811156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:38.811156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:38.811156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:38.811156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:38.811156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:38.811156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:38.811156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:38.811156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:38.812401 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:38.812401 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:38.812401 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:38.813575 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 336/657] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:38.814464 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df28f5d-b4df-4442-afd2-c2fa21330b82 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:38.897580 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b290f-ee47-4647-b886-c71672e3cabe None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:38.922046 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df28f5d-b4df-4442-afd2-c2fa21330b82 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:38.933347 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23001913-6d4f-417f-bc49-75d06442f69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:38.934262 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23001913-6d4f-417f-bc49-75d06442f69c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:38.934694 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23001913-6d4f-417f-bc49-75d06442f69c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:38.935121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23001913-6d4f-417f-bc49-75d06442f69c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:38.943163 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23001913-6d4f-417f-bc49-75d06442f69c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:38.945479 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23001913-6d4f-417f-bc49-75d06442f69c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:38.957675 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-23001913-6d4f-417f-bc49-75d06442f69c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 12:23:38.957675 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:38.957675 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:38.957675 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:38.957675 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:38.957675 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:38.957675 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:38.957675 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:38.957675 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:38.957675 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:38.957675 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:38.957675 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:38.957675 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:23:38.957675 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:23:38.957675 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:23:38.957675 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:23:38.958877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:38.958877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:38.958877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:23:38.958877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:38.958877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:23:38.958877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:38.958877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:38.958877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:38.958877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:38.958877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:38.958877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:38.958877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:38.958877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:38.958877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:38.958877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:38.958877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:38.958877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:38.958877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:38.960496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:38.960496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:38.960496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:23:38.960496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:23:38.960496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:23:38.960496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:23:38.960496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:23:38.960496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:23:38.960496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 12:23:38.960496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:38.961916 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e1c5a-3f54-446c-85a9-ae0f01b807e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:38.962655 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e1c5a-3f54-446c-85a9-ae0f01b807e0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:38.963032 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 322/658] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 12:23:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:38.964081 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23001913-6d4f-417f-bc49-75d06442f69c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:38.967460 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e1c5a-3f54-446c-85a9-ae0f01b807e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:38.967858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e1c5a-3f54-446c-85a9-ae0f01b807e0 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:38.969098 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e1c5a-3f54-446c-85a9-ae0f01b807e0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:38.969302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e1c5a-3f54-446c-85a9-ae0f01b807e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:38.970835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e1c5a-3f54-446c-85a9-ae0f01b807e0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:38.979514 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:38.994970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 337/659] 10.222.0.49 () {56 vars in 1137 bytes} [Wed Aug 10 12:23:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:38.994970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e1c5a-3f54-446c-85a9-ae0f01b807e0 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:39.055599 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:39.073909 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23001913-6d4f-417f-bc49-75d06442f69c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:39.107522 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e1c5a-3f54-446c-85a9-ae0f01b807e0 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:39.114328 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b4b8b3-314c-4074-93c0-8ca415e6fc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:39.115127 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4b8b3-314c-4074-93c0-8ca415e6fc32 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:39.115675 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4b8b3-314c-4074-93c0-8ca415e6fc32 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:39.116601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4b8b3-314c-4074-93c0-8ca415e6fc32 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:39.117466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b4b8b3-314c-4074-93c0-8ca415e6fc32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:39.117932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b4b8b3-314c-4074-93c0-8ca415e6fc32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:39.119479 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b4b8b3-314c-4074-93c0-8ca415e6fc32 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:39.133177 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 323/660] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:39.134330 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b4b8b3-314c-4074-93c0-8ca415e6fc32 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:39.145066 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbfd06-fda3-4e6d-a5b5-7f11e6948ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:39.145807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbfd06-fda3-4e6d-a5b5-7f11e6948ffc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:39.146184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbfd06-fda3-4e6d-a5b5-7f11e6948ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:39.146552 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbfd06-fda3-4e6d-a5b5-7f11e6948ffc None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:39.148168 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edbfd06-fda3-4e6d-a5b5-7f11e6948ffc None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:39.148577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edbfd06-fda3-4e6d-a5b5-7f11e6948ffc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:39.150318 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edbfd06-fda3-4e6d-a5b5-7f11e6948ffc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:39.168544 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 338/661] 10.222.0.49 () {56 vars in 1139 bytes} [Wed Aug 10 12:23:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:39.169612 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbfd06-fda3-4e6d-a5b5-7f11e6948ffc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:39.242407 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b4b8b3-314c-4074-93c0-8ca415e6fc32 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:39.263680 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:39.275802 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:39.279071 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbfd06-fda3-4e6d-a5b5-7f11e6948ffc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:39.284910 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b868e-4c58-4c52-bb44-f5fbf5b3aab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:39.284910 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b868e-4c58-4c52-bb44-f5fbf5b3aab1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:39.285251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b868e-4c58-4c52-bb44-f5fbf5b3aab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:39.285677 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b868e-4c58-4c52-bb44-f5fbf5b3aab1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:39.314896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d1924-848e-4490-99ef-681c9498c99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:39.315373 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d1924-848e-4490-99ef-681c9498c99d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:39.315801 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d1924-848e-4490-99ef-681c9498c99d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:39.316158 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d1924-848e-4490-99ef-681c9498c99d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:39.323868 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d1924-848e-4490-99ef-681c9498c99d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:39.325278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d1924-848e-4490-99ef-681c9498c99d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:39.336489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-246d1924-848e-4490-99ef-681c9498c99d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:39.336489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:39.336489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:39.336489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:39.336489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:39.336489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:39.336489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:39.336489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:39.336489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:39.336489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:39.336489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:39.336489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:39.336489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:23:39.336489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:23:39.336489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:23:39.337602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:23:39.337602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:39.337602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:39.337602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:39.337602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:39.337602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:39.337602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:39.337602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:39.337602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:39.337602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:39.337602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:39.337602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:39.337602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:39.337602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:39.337602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:39.337602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:39.337602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:39.338676 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:23:39.338676 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:23:39.338676 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:23:39.338676 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:23:39.338676 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:23:39.338676 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:23:39.338676 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:39.338676 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:39.340387 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 339/662] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:39.341312 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d1924-848e-4490-99ef-681c9498c99d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:39.361409 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b868e-4c58-4c52-bb44-f5fbf5b3aab1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:39.363099 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b868e-4c58-4c52-bb44-f5fbf5b3aab1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:39.431621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:39.443295 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d1924-848e-4490-99ef-681c9498c99d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:39.470043 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 324/663] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:23:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:23:39.471286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b868e-4c58-4c52-bb44-f5fbf5b3aab1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:39.479136 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7e006-faeb-4370-a4fd-04943c187517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:39.479850 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7e006-faeb-4370-a4fd-04943c187517 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:39.480194 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7e006-faeb-4370-a4fd-04943c187517 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:39.480553 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7e006-faeb-4370-a4fd-04943c187517 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:39.511610 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:39.568228 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7e006-faeb-4370-a4fd-04943c187517 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:39.571325 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7e006-faeb-4370-a4fd-04943c187517 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:39.595071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b868e-4c58-4c52-bb44-f5fbf5b3aab1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:39.635136 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1aeee-029d-430e-8070-800eee5d9399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:39.636064 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1aeee-029d-430e-8070-800eee5d9399 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:39.636543 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1aeee-029d-430e-8070-800eee5d9399 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:39.637014 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1aeee-029d-430e-8070-800eee5d9399 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:39.637970 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a1aeee-029d-430e-8070-800eee5d9399 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:39.638511 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a1aeee-029d-430e-8070-800eee5d9399 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:39.640156 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a1aeee-029d-430e-8070-800eee5d9399 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:39.653762 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 325/664] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:23:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:39.654995 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1aeee-029d-430e-8070-800eee5d9399 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:39.680953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 340/665] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:23:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:23:39.681949 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7e006-faeb-4370-a4fd-04943c187517 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:39.683140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:39.743096 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:39.766579 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1aeee-029d-430e-8070-800eee5d9399 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:39.786026 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7e006-faeb-4370-a4fd-04943c187517 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:39.802734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af3c34-1880-4edc-8d75-265124c3078e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:39.804366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af3c34-1880-4edc-8d75-265124c3078e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:39.804848 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af3c34-1880-4edc-8d75-265124c3078e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:39.805331 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af3c34-1880-4edc-8d75-265124c3078e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:39.815886 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24af3c34-1880-4edc-8d75-265124c3078e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:39.817640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24af3c34-1880-4edc-8d75-265124c3078e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:39.822427 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49aa22-17cf-4059-a314-7fca74d573a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:39.822976 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49aa22-17cf-4059-a314-7fca74d573a9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:39.823284 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49aa22-17cf-4059-a314-7fca74d573a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:39.823562 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49aa22-17cf-4059-a314-7fca74d573a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:39.824277 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49aa22-17cf-4059-a314-7fca74d573a9 None admin] RBAC: Adding query filter params (role.id=a9bbb011afca4176a3def9356cb5f165, scope.project.id=8e201e5f281a4977be503e22daf149d7, user.id=eb3f6deffb54467f8b923956ed2a6acf) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:39.824585 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49aa22-17cf-4059-a314-7fca74d573a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:39.825997 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49aa22-17cf-4059-a314-7fca74d573a9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:39.833366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 341/666] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:23:39 2022] GET /identity/v3/role_assignments?user.id=eb3f6deffb54467f8b923956ed2a6acf&scope.project.id=8e201e5f281a4977be503e22daf149d7&role.id=a9bbb011afca4176a3def9356cb5f165 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:39.834356 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49aa22-17cf-4059-a314-7fca74d573a9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:39.836895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-24af3c34-1880-4edc-8d75-265124c3078e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:39.836895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:39.836895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:39.836895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:39.836895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:39.836895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:39.836895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:39.836895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:39.836895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:39.836895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:39.836895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:39.836895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:39.836895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:39.836895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:39.836895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:39.836895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:39.837830 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:39.837830 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:39.837830 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:39.837830 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:39.837830 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:39.837830 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:39.837830 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:39.837830 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:39.837830 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:39.837830 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:39.837830 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:39.837830 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:39.837830 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:39.837830 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:39.837830 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:39.837830 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:39.837830 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:39.838727 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:39.838727 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:39.838727 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:39.840928 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 326/667] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:39.846655 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af3c34-1880-4edc-8d75-265124c3078e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:39.929424 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:39.948469 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49aa22-17cf-4059-a314-7fca74d573a9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:39.958825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af3c34-1880-4edc-8d75-265124c3078e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:39.987426 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:39.997953 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb5f80-6d3b-4fa2-9984-57a4be7de718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:39.998917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb5f80-6d3b-4fa2-9984-57a4be7de718 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:39.999368 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb5f80-6d3b-4fa2-9984-57a4be7de718 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:39.999704 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb5f80-6d3b-4fa2-9984-57a4be7de718 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:40.000417 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb5f80-6d3b-4fa2-9984-57a4be7de718 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:40.000787 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb5f80-6d3b-4fa2-9984-57a4be7de718 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:40.002258 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb5f80-6d3b-4fa2-9984-57a4be7de718 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:40.006025 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70301404-cb9d-43c4-9f4c-9e0109424890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:40.006726 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70301404-cb9d-43c4-9f4c-9e0109424890 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:40.007006 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70301404-cb9d-43c4-9f4c-9e0109424890 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:40.007315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70301404-cb9d-43c4-9f4c-9e0109424890 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:40.017593 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 327/668] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:40.023531 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb5f80-6d3b-4fa2-9984-57a4be7de718 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:40.036763 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70301404-cb9d-43c4-9f4c-9e0109424890 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:40.038813 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70301404-cb9d-43c4-9f4c-9e0109424890 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:40.059858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-70301404-cb9d-43c4-9f4c-9e0109424890 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:40.059858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:40.059858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:40.059858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:40.059858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:40.059858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:40.059858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:40.059858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:40.059858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:40.059858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:40.059858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:40.059858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:40.059858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:40.059858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:40.059858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:40.059858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:40.060877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:40.060877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:40.060877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:40.060877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:40.060877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:40.060877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:40.060877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:40.060877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:40.060877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:40.060877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:40.060877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:40.060877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:40.060877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:40.060877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:40.060877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:40.060877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:40.060877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:40.061770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:40.061770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:40.061770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:40.061770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 342/669] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:40.072161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70301404-cb9d-43c4-9f4c-9e0109424890 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:40.153435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb5f80-6d3b-4fa2-9984-57a4be7de718 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:40.191446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70301404-cb9d-43c4-9f4c-9e0109424890 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:40.195545 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f9594-904e-4124-b9c9-69691f4b555d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:40.196504 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f9594-904e-4124-b9c9-69691f4b555d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:40.197050 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f9594-904e-4124-b9c9-69691f4b555d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:40.197539 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f9594-904e-4124-b9c9-69691f4b555d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:40.199849 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:40.206223 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f9594-904e-4124-b9c9-69691f4b555d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:40.208742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f9594-904e-4124-b9c9-69691f4b555d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:40.225679 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-ee3f9594-904e-4124-b9c9-69691f4b555d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:40.225679 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:40.225679 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:40.225679 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:40.225679 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:40.225679 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:40.225679 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:40.225679 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:40.225679 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:40.225679 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:40.225679 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:40.225679 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:40.225679 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:23:40.225679 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:23:40.225679 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:23:40.226761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:23:40.226761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:40.226761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:40.226761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:40.226761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:40.226761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:40.226761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:40.226761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:40.226761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:40.226761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:40.226761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:40.226761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:40.226761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:40.226761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:40.226761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:40.226761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:40.226761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:40.227681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:23:40.227681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:23:40.227681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:23:40.227681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:23:40.227681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:23:40.227681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:23:40.227681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:40.227681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:40.228910 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 328/670] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:23:40.234682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f9594-904e-4124-b9c9-69691f4b555d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:40.243955 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92fad35-ffd7-4ea9-9767-b7c7a1f2c353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:40.246504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fad35-ffd7-4ea9-9767-b7c7a1f2c353 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:40.247000 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fad35-ffd7-4ea9-9767-b7c7a1f2c353 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:40.247535 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fad35-ffd7-4ea9-9767-b7c7a1f2c353 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:40.248878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92fad35-ffd7-4ea9-9767-b7c7a1f2c353 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:40.249943 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92fad35-ffd7-4ea9-9767-b7c7a1f2c353 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:40.251968 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92fad35-ffd7-4ea9-9767-b7c7a1f2c353 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:40.273183 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 343/671] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:40.280718 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92fad35-ffd7-4ea9-9767-b7c7a1f2c353 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:40.330388 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:40.372187 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f9594-904e-4124-b9c9-69691f4b555d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:40.408142 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fafa58-f9c8-4b9c-9fc9-576966696eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:40.408973 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fafa58-f9c8-4b9c-9fc9-576966696eac None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:40.409398 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fafa58-f9c8-4b9c-9fc9-576966696eac None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:40.409825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fafa58-f9c8-4b9c-9fc9-576966696eac None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:40.410755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fafa58-f9c8-4b9c-9fc9-576966696eac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:40.411245 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fafa58-f9c8-4b9c-9fc9-576966696eac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:40.412940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fafa58-f9c8-4b9c-9fc9-576966696eac None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:40.421461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92fad35-ffd7-4ea9-9767-b7c7a1f2c353 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:40.427384 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 329/672] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:23:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:40.429721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0120b14-eabc-4c66-8232-fbf98b3be490 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:40.430213 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0120b14-eabc-4c66-8232-fbf98b3be490 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:40.430755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0120b14-eabc-4c66-8232-fbf98b3be490 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:40.436383 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 330/673] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:40.437414 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0120b14-eabc-4c66-8232-fbf98b3be490 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:40.463960 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18300e47-1300-4afb-8b11-3ad5c64bb1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:40.464845 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18300e47-1300-4afb-8b11-3ad5c64bb1ae None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:40.465131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18300e47-1300-4afb-8b11-3ad5c64bb1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:40.465410 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18300e47-1300-4afb-8b11-3ad5c64bb1ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:40.475818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18300e47-1300-4afb-8b11-3ad5c64bb1ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:40.477605 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18300e47-1300-4afb-8b11-3ad5c64bb1ae None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:40.490431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-18300e47-1300-4afb-8b11-3ad5c64bb1ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:40.490431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:40.490431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:40.490431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:40.490431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:40.490431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:40.490431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:40.490431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:40.490431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:40.490431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:40.490431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:40.490431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:40.490431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:23:40.490431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:23:40.490431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:23:40.492108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:23:40.492108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:40.492108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:40.492108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:40.492108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:40.492108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:40.492108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:40.492108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:40.492108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:40.492108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:40.492108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:40.492108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:40.492108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:40.492108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:40.492108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:40.492108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:40.492108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:40.492983 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:23:40.492983 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:23:40.492983 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:23:40.492983 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:23:40.492983 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:23:40.492983 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:23:40.492983 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:40.492983 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:40.495386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 344/674] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:40.497036 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f9f0f-85cb-4530-9faa-8b7013ee425b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:40.497614 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f9f0f-85cb-4530-9faa-8b7013ee425b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:40.498084 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f9f0f-85cb-4530-9faa-8b7013ee425b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:40.514883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:40.517674 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-888f9f0f-85cb-4530-9faa-8b7013ee425b None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:40.519109 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-888f9f0f-85cb-4530-9faa-8b7013ee425b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:40.580989 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0120b14-eabc-4c66-8232-fbf98b3be490 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:40.595326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 345/675] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:40.596199 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f9f0f-85cb-4530-9faa-8b7013ee425b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:40.619068 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff8871a-0b79-4338-b78a-1422b2a25f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:40.619779 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8871a-0b79-4338-b78a-1422b2a25f7b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:40.619989 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8871a-0b79-4338-b78a-1422b2a25f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:40.620264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8871a-0b79-4338-b78a-1422b2a25f7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:40.621255 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff8871a-0b79-4338-b78a-1422b2a25f7b None admin] RBAC: Adding query filter params (role.id=a9bbb011afca4176a3def9356cb5f165, scope.project.id=8e201e5f281a4977be503e22daf149d7, user.id=35dd27fd356241b885706375d880222f) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:40.621568 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff8871a-0b79-4338-b78a-1422b2a25f7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:40.622992 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff8871a-0b79-4338-b78a-1422b2a25f7b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:40.629185 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 331/676] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:23:40 2022] GET /identity/v3/role_assignments?user.id=35dd27fd356241b885706375d880222f&scope.project.id=8e201e5f281a4977be503e22daf149d7&role.id=a9bbb011afca4176a3def9356cb5f165 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:40.630275 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f405c-11fe-4845-b671-a158f162979f None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:40.630561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f405c-11fe-4845-b671-a158f162979f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:40.630938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f405c-11fe-4845-b671-a158f162979f None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:40.633106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 332/677] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:40.633375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f405c-11fe-4845-b671-a158f162979f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:40.639507 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:40.746268 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f9f0f-85cb-4530-9faa-8b7013ee425b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:40.769277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f405c-11fe-4845-b671-a158f162979f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:40.792167 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a703a-d37a-44a7-8d2a-c12ea9574d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:40.792793 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a703a-d37a-44a7-8d2a-c12ea9574d96 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:40.793075 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a703a-d37a-44a7-8d2a-c12ea9574d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:40.793350 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a703a-d37a-44a7-8d2a-c12ea9574d96 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:40.793734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:40.794046 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a703a-d37a-44a7-8d2a-c12ea9574d96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:40.794355 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a703a-d37a-44a7-8d2a-c12ea9574d96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:40.795864 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a703a-d37a-44a7-8d2a-c12ea9574d96 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:40.810953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 346/678] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:23:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:40.812310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dda61c-d1b7-446a-a9fd-0bf02991d70e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:40.812604 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dda61c-d1b7-446a-a9fd-0bf02991d70e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:40.812879 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dda61c-d1b7-446a-a9fd-0bf02991d70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:40.816052 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9cc29f-c9fc-403f-8f69-2211e930cc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:40.817021 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9cc29f-c9fc-403f-8f69-2211e930cc10 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:40.817302 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9cc29f-c9fc-403f-8f69-2211e930cc10 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:40.817569 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9cc29f-c9fc-403f-8f69-2211e930cc10 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:40.818207 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9cc29f-c9fc-403f-8f69-2211e930cc10 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:40.821045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9cc29f-c9fc-403f-8f69-2211e930cc10 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:40.832546 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-28dda61c-d1b7-446a-a9fd-0bf02991d70e None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:40.833719 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-28dda61c-d1b7-446a-a9fd-0bf02991d70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:40.834915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-cd9cc29f-c9fc-403f-8f69-2211e930cc10 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 12:23:40.834915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:40.834915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:23:40.834915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:23:40.834915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:23:40.834915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:23:40.834915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:23:40.834915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:23:40.834915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 12:23:40.834915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:40.834915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:23:40.834915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:40.834915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:40.834915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:40.834915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:40.834915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:40.834915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:40.836074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:40.836074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:40.836074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:40.836074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:40.836074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:40.836074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:40.836074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:23:40.836074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:23:40.836074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:23:40.836074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:23:40.836074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:40.836074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:40.836074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:40.836074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:40.836074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:40.836074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:40.836940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:40.836940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:40.836940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:40.836940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:40.836940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:40.836940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:40.836940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:40.836940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:40.836940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:40.836940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:40.836940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:23:40.836940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:23:40.836940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 12:23:40.836940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:40.839518 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 333/679] 10.222.0.49 () {56 vars in 1129 bytes} [Wed Aug 10 12:23:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:40.840091 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9cc29f-c9fc-403f-8f69-2211e930cc10 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:40.925627 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 347/680] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:40.927125 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dda61c-d1b7-446a-a9fd-0bf02991d70e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:40.946092 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:40.971074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9cc29f-c9fc-403f-8f69-2211e930cc10 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:41.012861 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61976190-200e-413b-aca4-9ef9756a6ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:41.013495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61976190-200e-413b-aca4-9ef9756a6ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:41.013781 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61976190-200e-413b-aca4-9ef9756a6ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:41.014054 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61976190-200e-413b-aca4-9ef9756a6ae8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:41.014801 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61976190-200e-413b-aca4-9ef9756a6ae8 None admin] RBAC: Adding query filter params (role.id=a9bbb011afca4176a3def9356cb5f165, scope.project.id=8e201e5f281a4977be503e22daf149d7, user.id=5db814952f9f466abf6148b7a0db2173) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:41.015112 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61976190-200e-413b-aca4-9ef9756a6ae8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:41.016734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61976190-200e-413b-aca4-9ef9756a6ae8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:41.025021 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 334/681] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:23:40 2022] GET /identity/v3/role_assignments?user.id=5db814952f9f466abf6148b7a0db2173&scope.project.id=8e201e5f281a4977be503e22daf149d7&role.id=a9bbb011afca4176a3def9356cb5f165 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:41.025972 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61976190-200e-413b-aca4-9ef9756a6ae8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:41.133496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:41.163045 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dda61c-d1b7-446a-a9fd-0bf02991d70e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:41.201656 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c114e1-2fae-485a-a117-b97ff4370901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:41.202274 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c114e1-2fae-485a-a117-b97ff4370901 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:41.202557 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c114e1-2fae-485a-a117-b97ff4370901 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:41.202829 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c114e1-2fae-485a-a117-b97ff4370901 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:41.203504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c114e1-2fae-485a-a117-b97ff4370901 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:41.203807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c114e1-2fae-485a-a117-b97ff4370901 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:41.205195 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c114e1-2fae-485a-a117-b97ff4370901 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:41.211244 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 348/682] 10.222.0.49 () {56 vars in 1124 bytes} [Wed Aug 10 12:23:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:41.216337 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c114e1-2fae-485a-a117-b97ff4370901 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:41.234813 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61976190-200e-413b-aca4-9ef9756a6ae8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:41.313209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db20eb-f48f-41d3-a383-436e6f9bbb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:41.313848 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db20eb-f48f-41d3-a383-436e6f9bbb2a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:41.314131 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db20eb-f48f-41d3-a383-436e6f9bbb2a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:41.314402 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db20eb-f48f-41d3-a383-436e6f9bbb2a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:41.331694 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db20eb-f48f-41d3-a383-436e6f9bbb2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:41.339235 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-09db20eb-f48f-41d3-a383-436e6f9bbb2a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:41.339235 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:41.339235 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:41.339235 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:41.339235 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:41.339235 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:41.339235 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:41.339235 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:41.339235 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:41.339235 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:41.339235 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:41.339235 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:41.339235 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:41.339235 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:41.339235 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:41.339235 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:41.340335 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:41.340335 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:41.340335 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:41.340335 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:41.340335 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:41.340335 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:41.340335 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:41.340335 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:41.340335 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:41.340335 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:41.340335 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:41.340335 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:41.340335 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:41.340335 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:41.340335 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:41.340335 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:41.341184 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:41.341184 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:41.341184 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:41.341184 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:41.341184 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:41.341184 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:41.341184 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:41.341184 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:41.341184 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:41.341184 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:41.341184 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:41.341184 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:41.341184 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 335/683] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 12:23:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:41.351268 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db20eb-f48f-41d3-a383-436e6f9bbb2a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:41.355611 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:41.378666 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c114e1-2fae-485a-a117-b97ff4370901 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:41.418662 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad1450-d75e-4ce3-b306-52bea7bda40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:41.419323 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad1450-d75e-4ce3-b306-52bea7bda40a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:41.419605 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad1450-d75e-4ce3-b306-52bea7bda40a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:41.419882 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad1450-d75e-4ce3-b306-52bea7bda40a None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:41.420511 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad1450-d75e-4ce3-b306-52bea7bda40a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:41.420816 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad1450-d75e-4ce3-b306-52bea7bda40a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:41.422183 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad1450-d75e-4ce3-b306-52bea7bda40a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:41.428221 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 349/684] 10.222.0.49 () {56 vars in 1124 bytes} [Wed Aug 10 12:23:41 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:23:41.479084 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db20eb-f48f-41d3-a383-436e6f9bbb2a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:41.487554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:41.518714 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59697594-bb26-47ff-8572-124a5173aebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:41.519381 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59697594-bb26-47ff-8572-124a5173aebf None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:41.519662 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59697594-bb26-47ff-8572-124a5173aebf None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:41.519938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59697594-bb26-47ff-8572-124a5173aebf None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:41.520619 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59697594-bb26-47ff-8572-124a5173aebf None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:41.520925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59697594-bb26-47ff-8572-124a5173aebf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:41.522983 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59697594-bb26-47ff-8572-124a5173aebf None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:41.532783 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 336/685] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 12:23:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:23:41.539920 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad1450-d75e-4ce3-b306-52bea7bda40a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:41.653849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad1450-d75e-4ce3-b306-52bea7bda40a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:41.689078 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42bf32-8b28-4659-9edd-742ca260ffb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:41.689710 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42bf32-8b28-4659-9edd-742ca260ffb5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:41.689990 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42bf32-8b28-4659-9edd-742ca260ffb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:41.690260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42bf32-8b28-4659-9edd-742ca260ffb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:41.701705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42bf32-8b28-4659-9edd-742ca260ffb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:41.703146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42bf32-8b28-4659-9edd-742ca260ffb5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:41.705699 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:41.717785 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-dd42bf32-8b28-4659-9edd-742ca260ffb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:41.717785 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:41.717785 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:41.717785 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:41.717785 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:41.717785 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:41.717785 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:41.717785 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:41.717785 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:41.717785 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:41.717785 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:41.717785 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:41.717785 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:41.717785 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:41.717785 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:41.717785 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:41.728577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:41.728577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:41.728577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:41.728577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:41.728577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:41.728577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:41.728577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:41.728577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:41.728577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:41.728577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:41.728577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:41.728577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:41.728577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:41.728577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:41.728577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:41.728577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:41.728577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:41.729502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:41.729502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:41.729502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:41.729502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 350/686] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:41.729740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59697594-bb26-47ff-8572-124a5173aebf None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:41.855060 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59697594-bb26-47ff-8572-124a5173aebf None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:41.898619 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757aab78-44a2-484a-9b28-dd640ed50655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:41.899258 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757aab78-44a2-484a-9b28-dd640ed50655 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:41.899542 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757aab78-44a2-484a-9b28-dd640ed50655 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:41.899816 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757aab78-44a2-484a-9b28-dd640ed50655 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:41.900456 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757aab78-44a2-484a-9b28-dd640ed50655 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:41.900761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757aab78-44a2-484a-9b28-dd640ed50655 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:41.902126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757aab78-44a2-484a-9b28-dd640ed50655 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:41.924519 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 337/687] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:41.930599 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42bf32-8b28-4659-9edd-742ca260ffb5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:42.051829 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42bf32-8b28-4659-9edd-742ca260ffb5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:42.100603 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3285c9-67f8-466e-b719-2fa34daa3626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:42.101847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3285c9-67f8-466e-b719-2fa34daa3626 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:42.102347 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3285c9-67f8-466e-b719-2fa34daa3626 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:42.102820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3285c9-67f8-466e-b719-2fa34daa3626 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:42.110657 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3285c9-67f8-466e-b719-2fa34daa3626 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:42.112353 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3285c9-67f8-466e-b719-2fa34daa3626 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:42.131157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-2c3285c9-67f8-466e-b719-2fa34daa3626 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 12:23:42.131157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:42.131157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:42.131157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:42.131157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:42.131157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:42.131157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:42.131157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:42.131157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:42.131157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:42.131157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:42.131157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:42.131157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:23:42.131157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:23:42.131157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:23:42.131157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:23:42.132196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:42.132196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:42.132196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:23:42.132196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:42.132196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:23:42.132196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:42.132196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:42.132196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:42.132196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:42.132196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:42.132196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:42.132196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:42.132196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:42.132196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:42.132196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:42.132196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:42.132196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:42.132196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:42.133132 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:42.133132 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:42.133132 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:23:42.133132 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:23:42.133132 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:23:42.133132 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:23:42.133132 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:23:42.133132 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:23:42.133132 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 12:23:42.133132 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:42.135646 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 351/688] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:23:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:42.141070 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757aab78-44a2-484a-9b28-dd640ed50655 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:42.204237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:42.296384 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757aab78-44a2-484a-9b28-dd640ed50655 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:42.344610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d995cc-b2af-460d-9b67-937f16734f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:42.345233 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d995cc-b2af-460d-9b67-937f16734f34 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:42.345515 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d995cc-b2af-460d-9b67-937f16734f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:42.345790 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d995cc-b2af-460d-9b67-937f16734f34 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:42.346520 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d995cc-b2af-460d-9b67-937f16734f34 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:42.346829 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d995cc-b2af-460d-9b67-937f16734f34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:42.348297 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d995cc-b2af-460d-9b67-937f16734f34 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:42.372740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 338/689] 10.222.0.49 () {56 vars in 1135 bytes} [Wed Aug 10 12:23:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:42.377929 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3285c9-67f8-466e-b719-2fa34daa3626 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:42.486557 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:42.547204 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3285c9-67f8-466e-b719-2fa34daa3626 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:42.602376 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bbce5d-98ea-4c49-8fad-507de70f9707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:42.603450 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbce5d-98ea-4c49-8fad-507de70f9707 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:42.603953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbce5d-98ea-4c49-8fad-507de70f9707 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:42.604432 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbce5d-98ea-4c49-8fad-507de70f9707 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:42.679159 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bbce5d-98ea-4c49-8fad-507de70f9707 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:42.681005 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bbce5d-98ea-4c49-8fad-507de70f9707 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:42.683718 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc9541-6d7d-403c-9866-c4f5243b5cee None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:42.684006 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc9541-6d7d-403c-9866-c4f5243b5cee None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:42.684284 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc9541-6d7d-403c-9866-c4f5243b5cee None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:42.685856 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 339/690] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:42.692420 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5b012-fc59-41ca-b890-8f5f228472f6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:42.692706 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5b012-fc59-41ca-b890-8f5f228472f6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:42.692982 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5b012-fc59-41ca-b890-8f5f228472f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:42.727032 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-40c5b012-fc59-41ca-b890-8f5f228472f6 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:42.730483 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-40c5b012-fc59-41ca-b890-8f5f228472f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:42.770166 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:42.856524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 340/691] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:42.867429 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 352/692] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:23:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:23:42.872958 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5b012-fc59-41ca-b890-8f5f228472f6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:42.892286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:42.894224 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bbce5d-98ea-4c49-8fad-507de70f9707 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:43.019546 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5b012-fc59-41ca-b890-8f5f228472f6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:43.035047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bbce5d-98ea-4c49-8fad-507de70f9707 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:43.092753 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05310e1-9a7e-4ef7-bf35-2e817b0b1fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:43.093394 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05310e1-9a7e-4ef7-bf35-2e817b0b1fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:43.093674 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05310e1-9a7e-4ef7-bf35-2e817b0b1fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:43.093956 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05310e1-9a7e-4ef7-bf35-2e817b0b1fa9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:43.098674 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e2258a-f022-47c9-81b1-6a3dda3fe958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:43.099349 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2258a-f022-47c9-81b1-6a3dda3fe958 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:43.099632 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2258a-f022-47c9-81b1-6a3dda3fe958 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:43.099910 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2258a-f022-47c9-81b1-6a3dda3fe958 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:43.105730 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05310e1-9a7e-4ef7-bf35-2e817b0b1fa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:43.107362 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05310e1-9a7e-4ef7-bf35-2e817b0b1fa9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:43.110818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e2258a-f022-47c9-81b1-6a3dda3fe958 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:43.115536 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-b5e2258a-f022-47c9-81b1-6a3dda3fe958 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:23:43.115536 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:43.115536 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:43.115536 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:43.115536 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:43.115536 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:43.115536 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:43.115536 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:43.115536 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:43.115536 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:43.115536 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:43.115536 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:43.115536 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:43.115536 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:43.115536 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:43.115536 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:43.117114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:43.117114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:43.117114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:43.117114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:43.117114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:43.117114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:43.117114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:43.117114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:43.117114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:43.117114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:43.117114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:43.117114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:43.117114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:43.117114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:43.117114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:43.117114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:43.117939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:43.117939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:43.117939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:43.117939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:43.117939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:43.117939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:43.117939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:43.117939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:43.117939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:43.117939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:43.117939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:23:43.117939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:43.117939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 353/693] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:23:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:43.119121 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb061fe-620c-409f-826b-de928c7b73e6 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:43.120201 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb061fe-620c-409f-826b-de928c7b73e6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:43.120201 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb061fe-620c-409f-826b-de928c7b73e6 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:43.121511 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 354/694] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:43.125100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-c05310e1-9a7e-4ef7-bf35-2e817b0b1fa9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:43.125100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:43.125100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:43.125100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:43.125100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:43.125100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:43.125100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:43.125100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:43.125100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:43.125100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:43.125100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:43.125100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:43.125100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:43.125100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:43.125100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:43.125100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:43.126469 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:43.126469 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:43.126469 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:43.126469 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:43.126469 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:43.126469 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:43.126469 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:43.126469 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:43.126469 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:43.126469 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:43.126469 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:43.126469 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:43.126469 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:43.126469 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:43.126469 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:43.126469 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:43.126469 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:43.127611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:43.127611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:43.127611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:43.127783 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb061fe-620c-409f-826b-de928c7b73e6 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:43.129053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 341/695] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:23:43.130242 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2d425-7b8f-49f9-ade9-8ec0dcb6ce7b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:43.130530 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2d425-7b8f-49f9-ade9-8ec0dcb6ce7b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:43.130868 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2d425-7b8f-49f9-ade9-8ec0dcb6ce7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:43.149759 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:43.151860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0ec2d425-7b8f-49f9-ade9-8ec0dcb6ce7b None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:43.153067 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0ec2d425-7b8f-49f9-ade9-8ec0dcb6ce7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:43.237466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:43.251254 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 342/696] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:43.252450 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2d425-7b8f-49f9-ade9-8ec0dcb6ce7b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:43.262802 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb061fe-620c-409f-826b-de928c7b73e6 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:43.310149 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf292c2-f7e3-4edd-b517-7d2ff172da92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:43.311431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf292c2-f7e3-4edd-b517-7d2ff172da92 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:43.311948 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf292c2-f7e3-4edd-b517-7d2ff172da92 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:43.312520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf292c2-f7e3-4edd-b517-7d2ff172da92 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:43.313558 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf292c2-f7e3-4edd-b517-7d2ff172da92 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:43.314863 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf292c2-f7e3-4edd-b517-7d2ff172da92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:43.316499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf292c2-f7e3-4edd-b517-7d2ff172da92 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:43.328202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 355/697] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:23:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:43.329801 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf292c2-f7e3-4edd-b517-7d2ff172da92 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:43.398845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2d425-7b8f-49f9-ade9-8ec0dcb6ce7b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:43.436417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:43.440975 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d0c7d-0e79-46c6-bcc7-0fa01cc6cd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:43.442340 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d0c7d-0e79-46c6-bcc7-0fa01cc6cd01 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:43.443094 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d0c7d-0e79-46c6-bcc7-0fa01cc6cd01 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:43.443601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d0c7d-0e79-46c6-bcc7-0fa01cc6cd01 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:43.445121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d0c7d-0e79-46c6-bcc7-0fa01cc6cd01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:43.445890 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d0c7d-0e79-46c6-bcc7-0fa01cc6cd01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:43.448658 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d0c7d-0e79-46c6-bcc7-0fa01cc6cd01 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:43.461239 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf292c2-f7e3-4edd-b517-7d2ff172da92 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:43.463289 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 343/698] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:43.465098 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d0c7d-0e79-46c6-bcc7-0fa01cc6cd01 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:43.504179 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8659f06-69fc-4438-bc6c-6cd6d5dbb4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:43.505590 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8659f06-69fc-4438-bc6c-6cd6d5dbb4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:43.506367 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8659f06-69fc-4438-bc6c-6cd6d5dbb4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:43.506828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8659f06-69fc-4438-bc6c-6cd6d5dbb4d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:43.520566 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8659f06-69fc-4438-bc6c-6cd6d5dbb4d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:43.523122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8659f06-69fc-4438-bc6c-6cd6d5dbb4d0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:43.541083 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:43.542419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-f8659f06-69fc-4438-bc6c-6cd6d5dbb4d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:43.542419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:43.542419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:43.542419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:43.542419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:43.542419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:43.542419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:43.542419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:43.542419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:43.542419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:43.542419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:43.542419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:43.542419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:43.542419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:43.542419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:43.542419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:43.543374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:43.543374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:43.543374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:43.543374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:43.543374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:43.543374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:43.543374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:43.543374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:43.543374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:43.543374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:43.543374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:43.543374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:43.543374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:43.543374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:43.543374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:43.543374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:43.543374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:43.544291 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:43.544291 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:43.544291 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:43.546336 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 356/699] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:43.547436 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8659f06-69fc-4438-bc6c-6cd6d5dbb4d0 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:43.598537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d0c7d-0e79-46c6-bcc7-0fa01cc6cd01 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:43.643874 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c300b-2486-4849-b3da-6f22f5f5d7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:43.644673 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c300b-2486-4849-b3da-6f22f5f5d7b3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:43.645590 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c300b-2486-4849-b3da-6f22f5f5d7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:43.645974 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c300b-2486-4849-b3da-6f22f5f5d7b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:43.660178 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290c300b-2486-4849-b3da-6f22f5f5d7b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:43.662305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290c300b-2486-4849-b3da-6f22f5f5d7b3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:43.676889 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8659f06-69fc-4438-bc6c-6cd6d5dbb4d0 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:43.683009 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-290c300b-2486-4849-b3da-6f22f5f5d7b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:43.683009 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:43.683009 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:43.683009 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:43.683009 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:43.683009 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:43.683009 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:43.683009 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:43.683009 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:43.683009 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:43.683009 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:43.683009 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:43.683009 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:43.683009 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:43.683009 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:43.683009 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:43.684055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:43.684055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:43.684055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:43.684055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:43.684055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:43.684055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:43.684055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:43.684055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:43.684055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:43.684055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:43.684055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:43.684055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:43.684055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:43.684055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:43.684055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:43.684055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:43.684055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:43.684988 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:43.684988 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:43.684988 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:43.684988 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 344/700] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:43.685956 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c300b-2486-4849-b3da-6f22f5f5d7b3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:43.714043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:43.722164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ffeabc-91d9-4b01-99c1-10322fcef353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:43.723480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffeabc-91d9-4b01-99c1-10322fcef353 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:43.723918 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffeabc-91d9-4b01-99c1-10322fcef353 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:43.724618 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffeabc-91d9-4b01-99c1-10322fcef353 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:43.734790 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ffeabc-91d9-4b01-99c1-10322fcef353 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:43.736532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ffeabc-91d9-4b01-99c1-10322fcef353 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:43.753345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-00ffeabc-91d9-4b01-99c1-10322fcef353 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:43.753345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:43.753345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:43.753345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:43.753345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:43.753345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:43.753345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:43.753345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:43.753345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:43.753345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:43.753345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:43.753345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:43.753345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:23:43.753345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:23:43.753345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:23:43.754487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:23:43.754487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:43.754487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:43.754487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:43.754487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:43.754487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:43.754487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:43.754487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:43.754487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:43.754487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:43.754487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:43.754487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:43.754487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:43.754487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:43.754487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:43.754487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:43.754487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:43.755700 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:23:43.755700 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:23:43.755700 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:23:43.755700 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:23:43.755700 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:23:43.755700 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:23:43.755700 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:43.755700 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:43.758748 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 357/701] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:43.760179 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ffeabc-91d9-4b01-99c1-10322fcef353 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:43.813130 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c300b-2486-4849-b3da-6f22f5f5d7b3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:43.825967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:43.856451 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4baa4-fc41-417c-b475-211a618fed15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:43.857216 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4baa4-fc41-417c-b475-211a618fed15 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:43.857626 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4baa4-fc41-417c-b475-211a618fed15 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:43.858018 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4baa4-fc41-417c-b475-211a618fed15 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:43.858749 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b4baa4-fc41-417c-b475-211a618fed15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:43.859138 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b4baa4-fc41-417c-b475-211a618fed15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:43.860634 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b4baa4-fc41-417c-b475-211a618fed15 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:43.872192 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ffeabc-91d9-4b01-99c1-10322fcef353 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:43.880475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 345/702] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:43.880475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4baa4-fc41-417c-b475-211a618fed15 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:43.912643 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341d270-771a-4c4c-aa12-3b87c48955c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:43.913443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341d270-771a-4c4c-aa12-3b87c48955c5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:43.913727 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341d270-771a-4c4c-aa12-3b87c48955c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:43.914007 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341d270-771a-4c4c-aa12-3b87c48955c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:43.914645 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341d270-771a-4c4c-aa12-3b87c48955c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:43.914951 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341d270-771a-4c4c-aa12-3b87c48955c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:43.916356 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341d270-771a-4c4c-aa12-3b87c48955c5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:43.927816 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 358/703] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:43.929804 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce2e70-ced3-4b3f-b718-63cf462e35fd None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:43.930178 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce2e70-ced3-4b3f-b718-63cf462e35fd None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:43.930427 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce2e70-ced3-4b3f-b718-63cf462e35fd None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:43.931587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 359/704] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:43.932806 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ad22e-2290-45de-80aa-557ddcaf197e None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:43.933109 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ad22e-2290-45de-80aa-557ddcaf197e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:43.933386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ad22e-2290-45de-80aa-557ddcaf197e None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:43.934565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 360/705] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:43.935410 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ad22e-2290-45de-80aa-557ddcaf197e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:43.979832 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4baa4-fc41-417c-b475-211a618fed15 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:43.989217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:44.014252 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b224989-ab73-4b44-9a54-b11d1f35d13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:44.014855 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b224989-ab73-4b44-9a54-b11d1f35d13e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:44.015141 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b224989-ab73-4b44-9a54-b11d1f35d13e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:44.015440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b224989-ab73-4b44-9a54-b11d1f35d13e None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:44.016118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b224989-ab73-4b44-9a54-b11d1f35d13e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:44.016424 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b224989-ab73-4b44-9a54-b11d1f35d13e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:44.017849 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b224989-ab73-4b44-9a54-b11d1f35d13e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:44.029321 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 346/706] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:23:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:44.030106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b224989-ab73-4b44-9a54-b11d1f35d13e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:44.042218 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ad22e-2290-45de-80aa-557ddcaf197e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:44.059717 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:44.076121 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43c624-e463-48ec-b787-73c08c20ec2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:44.076662 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43c624-e463-48ec-b787-73c08c20ec2f None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:44.076938 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43c624-e463-48ec-b787-73c08c20ec2f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:44.077209 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43c624-e463-48ec-b787-73c08c20ec2f None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:44.077839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43c624-e463-48ec-b787-73c08c20ec2f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:44.079354 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43c624-e463-48ec-b787-73c08c20ec2f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:44.128267 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b224989-ab73-4b44-9a54-b11d1f35d13e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:44.161165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebefadf-868f-4078-8be1-20eb97f49ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:44.161789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebefadf-868f-4078-8be1-20eb97f49ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:44.162069 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebefadf-868f-4078-8be1-20eb97f49ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:44.162337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebefadf-868f-4078-8be1-20eb97f49ad3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:44.162471 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 361/707] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 12:23:43 2022] POST /identity/v3/users => generated 254 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:23:44.169910 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebefadf-868f-4078-8be1-20eb97f49ad3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:44.173694 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebefadf-868f-4078-8be1-20eb97f49ad3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:44.176720 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93708e4-9dfb-4ce6-a486-5fc89c9afb90 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:44.177009 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93708e4-9dfb-4ce6-a486-5fc89c9afb90 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:44.177306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93708e4-9dfb-4ce6-a486-5fc89c9afb90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:44.185447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-1ebefadf-868f-4078-8be1-20eb97f49ad3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:23:44.185447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:44.185447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:44.185447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:44.185447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:44.185447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:44.185447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:44.185447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:44.185447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:44.185447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:44.185447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:44.185447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:44.185447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:23:44.185447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:23:44.185447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:23:44.185447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:23:44.186521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:44.186521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:44.186521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:23:44.186521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:44.186521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:23:44.186521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:44.186521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:44.186521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:44.186521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:44.186521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:44.186521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:44.186521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:44.186521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:44.186521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:44.186521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:44.186521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:44.186521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:44.186521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:44.187517 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:44.187517 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:44.187517 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:23:44.187517 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:23:44.187517 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:23:44.187517 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:23:44.187517 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:23:44.187517 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:23:44.187517 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:23:44.187517 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:44.187517 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 347/708] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Aug 10 12:23:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:44.189811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c1022-b8c2-4866-957f-a723dcb448cc None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:44.189811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c1022-b8c2-4866-957f-a723dcb448cc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:44.189811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c1022-b8c2-4866-957f-a723dcb448cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:44.207928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e93708e4-9dfb-4ce6-a486-5fc89c9afb90 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:44.209147 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-452c1022-b8c2-4866-957f-a723dcb448cc None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:44.209263 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e93708e4-9dfb-4ce6-a486-5fc89c9afb90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:44.211207 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-452c1022-b8c2-4866-957f-a723dcb448cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:44.295605 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 362/709] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:44.296471 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93708e4-9dfb-4ce6-a486-5fc89c9afb90 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:44.296952 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 348/710] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:44.297946 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c1022-b8c2-4866-957f-a723dcb448cc None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:44.338116 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:44.351940 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:44.423951 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93708e4-9dfb-4ce6-a486-5fc89c9afb90 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:44.425538 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c1022-b8c2-4866-957f-a723dcb448cc None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:44.459812 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803db995-82d4-488e-a047-c8193e27fb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:44.460418 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803db995-82d4-488e-a047-c8193e27fb60 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:44.460697 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803db995-82d4-488e-a047-c8193e27fb60 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:44.461016 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803db995-82d4-488e-a047-c8193e27fb60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:44.461679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803db995-82d4-488e-a047-c8193e27fb60 None admin] RBAC: Adding query filter params (role.id=a9bbb011afca4176a3def9356cb5f165, scope.project.id=8e201e5f281a4977be503e22daf149d7, user.id=59395bfa505c4540b3957deee78eb698) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:44.461988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803db995-82d4-488e-a047-c8193e27fb60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:44.462149 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d38b8d-a1d6-4e4f-9ffa-7923052af185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:44.462742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d38b8d-a1d6-4e4f-9ffa-7923052af185 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:44.463020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d38b8d-a1d6-4e4f-9ffa-7923052af185 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:44.463323 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d38b8d-a1d6-4e4f-9ffa-7923052af185 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:44.463431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803db995-82d4-488e-a047-c8193e27fb60 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:44.464044 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d38b8d-a1d6-4e4f-9ffa-7923052af185 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:44.464347 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d38b8d-a1d6-4e4f-9ffa-7923052af185 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:44.465764 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d38b8d-a1d6-4e4f-9ffa-7923052af185 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:44.475890 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 363/711] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:23:44 2022] GET /identity/v3/role_assignments?user.id=59395bfa505c4540b3957deee78eb698&scope.project.id=8e201e5f281a4977be503e22daf149d7&role.id=a9bbb011afca4176a3def9356cb5f165 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:44.475890 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803db995-82d4-488e-a047-c8193e27fb60 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:44.489342 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 349/712] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Aug 10 12:23:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:44.490202 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d38b8d-a1d6-4e4f-9ffa-7923052af185 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:44.595977 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:44.618194 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:44.621604 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803db995-82d4-488e-a047-c8193e27fb60 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:44.621928 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d38b8d-a1d6-4e4f-9ffa-7923052af185 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:44.665560 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58835a5c-9e76-48bb-978b-399ccdbce496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:44.665940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58835a5c-9e76-48bb-978b-399ccdbce496 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:44.666235 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58835a5c-9e76-48bb-978b-399ccdbce496 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:44.666513 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58835a5c-9e76-48bb-978b-399ccdbce496 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:44.667166 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58835a5c-9e76-48bb-978b-399ccdbce496 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:44.668952 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58835a5c-9e76-48bb-978b-399ccdbce496 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:44.675582 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24803a3-c027-44ee-bdf4-95643c88f520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:44.676060 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24803a3-c027-44ee-bdf4-95643c88f520 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:44.677003 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24803a3-c027-44ee-bdf4-95643c88f520 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:44.677003 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24803a3-c027-44ee-bdf4-95643c88f520 None admin] PATH_INFO: `/v3/services/network` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:44.677247 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24803a3-c027-44ee-bdf4-95643c88f520 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:44.679496 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24803a3-c027-44ee-bdf4-95643c88f520 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:44.690749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-a24803a3-c027-44ee-bdf4-95643c88f520 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 12:23:44.690749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:44.690749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:23:44.690749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:23:44.690749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:23:44.690749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:23:44.690749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:23:44.690749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:23:44.690749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 12:23:44.690749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:44.690749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:23:44.690749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:44.690749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:44.690749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:44.690749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:44.690749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:44.690749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:44.696341 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:44.696341 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:44.696341 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:44.696341 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:44.696341 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:44.696341 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:44.696341 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:23:44.696341 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:23:44.696341 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:23:44.696341 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:23:44.696341 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:44.696341 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:44.696341 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:44.696341 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:44.696341 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:44.696341 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:44.697369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:44.697369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:44.697369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:44.697369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:44.697369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:44.697369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:44.697369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:44.697369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:44.697369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:44.697369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:44.697369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:23:44.697369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:23:44.697369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 12:23:44.697369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:44.697369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 364/713] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:44 2022] GET /identity/v3/services/network => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:44.698347 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24803a3-c027-44ee-bdf4-95643c88f520 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:44.753121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 350/714] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 10 12:23:44 2022] POST /identity/v3/services => generated 251 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:23:44.754141 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58835a5c-9e76-48bb-978b-399ccdbce496 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:44.813234 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24803a3-c027-44ee-bdf4-95643c88f520 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:44.866386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce4933-8d38-4998-bcb9-704b6c65f978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:44.866922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce4933-8d38-4998-bcb9-704b6c65f978 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:44.867295 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce4933-8d38-4998-bcb9-704b6c65f978 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:44.867607 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce4933-8d38-4998-bcb9-704b6c65f978 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:44.869254 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58835a5c-9e76-48bb-978b-399ccdbce496 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:44.899594 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:44.938646 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f3ad6-acd6-4ed8-b3b8-a4054192a111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:44.939313 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f3ad6-acd6-4ed8-b3b8-a4054192a111 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:44.939597 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f3ad6-acd6-4ed8-b3b8-a4054192a111 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:44.939871 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f3ad6-acd6-4ed8-b3b8-a4054192a111 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:44.940512 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f3ad6-acd6-4ed8-b3b8-a4054192a111 None admin] RBAC: Adding query filter params (name=network) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:44.940817 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f3ad6-acd6-4ed8-b3b8-a4054192a111 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:44.942354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f3ad6-acd6-4ed8-b3b8-a4054192a111 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:44.943648 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce4933-8d38-4998-bcb9-704b6c65f978 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:44.945767 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce4933-8d38-4998-bcb9-704b6c65f978 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:44.948213 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 351/715] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 10 12:23:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:44.956504 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f3ad6-acd6-4ed8-b3b8-a4054192a111 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:44.960072 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:45.055208 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 365/716] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:23:44 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:23:45.060681 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce4933-8d38-4998-bcb9-704b6c65f978 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:45.122266 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f3ad6-acd6-4ed8-b3b8-a4054192a111 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:45.143209 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:45.173299 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67510a9e-687c-4727-9258-fff53bb14075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:45.173965 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67510a9e-687c-4727-9258-fff53bb14075 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:45.174251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67510a9e-687c-4727-9258-fff53bb14075 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:45.174531 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67510a9e-687c-4727-9258-fff53bb14075 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:45.175157 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67510a9e-687c-4727-9258-fff53bb14075 None admin] RBAC: Adding query filter params (type=network) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:45.175500 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67510a9e-687c-4727-9258-fff53bb14075 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:45.176871 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67510a9e-687c-4727-9258-fff53bb14075 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:45.177828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce4933-8d38-4998-bcb9-704b6c65f978 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:45.191374 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 352/717] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 10 12:23:44 2022] GET /identity/v3/services?type=network => generated 124 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:45.219541 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62e7d9-b044-4ca6-9642-23dc6055bee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:45.220191 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62e7d9-b044-4ca6-9642-23dc6055bee1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:45.220467 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62e7d9-b044-4ca6-9642-23dc6055bee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:45.220736 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62e7d9-b044-4ca6-9642-23dc6055bee1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:45.233045 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd62e7d9-b044-4ca6-9642-23dc6055bee1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:45.234617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd62e7d9-b044-4ca6-9642-23dc6055bee1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:45.250912 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-cd62e7d9-b044-4ca6-9642-23dc6055bee1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:45.250912 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:45.250912 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:45.250912 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:45.250912 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:45.250912 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:45.250912 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:45.250912 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:45.250912 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:45.250912 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:45.250912 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:45.250912 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:45.250912 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:45.250912 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:45.250912 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:45.250912 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:45.251964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:45.251964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:45.251964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:45.251964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:45.251964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:45.251964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:45.251964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:45.251964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:45.251964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:45.251964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:45.251964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:45.251964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:45.251964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:45.251964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:45.251964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:45.251964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:45.251964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:45.252859 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:45.252859 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:45.252859 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:45.252859 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 366/718] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:45.259337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67510a9e-687c-4727-9258-fff53bb14075 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:45.339943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:45.390414 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67510a9e-687c-4727-9258-fff53bb14075 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:45.431454 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b0b535-ac97-4eac-9587-bf25ad664fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:45.432088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0b535-ac97-4eac-9587-bf25ad664fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:45.432366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0b535-ac97-4eac-9587-bf25ad664fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:45.432639 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0b535-ac97-4eac-9587-bf25ad664fe0 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:45.433300 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b0b535-ac97-4eac-9587-bf25ad664fe0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:45.433604 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b0b535-ac97-4eac-9587-bf25ad664fe0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:45.434997 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b0b535-ac97-4eac-9587-bf25ad664fe0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:45.453843 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 353/719] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:45.455242 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62e7d9-b044-4ca6-9642-23dc6055bee1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:45.580629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62e7d9-b044-4ca6-9642-23dc6055bee1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:45.594248 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:45.616535 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b7173-076b-49b3-858b-8e608fcb2836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:45.619313 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b7173-076b-49b3-858b-8e608fcb2836 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:45.619313 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b7173-076b-49b3-858b-8e608fcb2836 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:45.619313 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b7173-076b-49b3-858b-8e608fcb2836 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:45.624858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b7173-076b-49b3-858b-8e608fcb2836 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:45.626297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b7173-076b-49b3-858b-8e608fcb2836 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:45.638356 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-b11b7173-076b-49b3-858b-8e608fcb2836 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:45.638356 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:45.638356 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:45.638356 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:45.638356 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:45.638356 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:45.638356 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:45.638356 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:45.638356 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:45.638356 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:45.638356 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:45.638356 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:45.638356 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:23:45.638356 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:23:45.638356 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:23:45.639287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:23:45.639287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:45.639287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:45.639287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:45.639287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:45.639287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:45.639287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:45.639287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:45.639287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:45.639287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:45.639287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:45.639287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:45.639287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:45.639287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:45.639287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:45.639287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:45.639287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:45.640192 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:23:45.640192 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:23:45.640192 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:23:45.640192 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:23:45.640192 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:23:45.640192 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:23:45.640192 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:45.640192 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:45.640192 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 367/720] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:45.645483 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b0b535-ac97-4eac-9587-bf25ad664fe0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:45.760936 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b0b535-ac97-4eac-9587-bf25ad664fe0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:45.788179 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:45.802291 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19f38f-f952-45ce-8817-a87a00c439cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:45.810179 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19f38f-f952-45ce-8817-a87a00c439cc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:45.810179 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19f38f-f952-45ce-8817-a87a00c439cc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:45.810179 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19f38f-f952-45ce-8817-a87a00c439cc None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:45.810179 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19f38f-f952-45ce-8817-a87a00c439cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:45.810179 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19f38f-f952-45ce-8817-a87a00c439cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:45.810179 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19f38f-f952-45ce-8817-a87a00c439cc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:45.825362 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 354/721] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:23:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:45.831416 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b7173-076b-49b3-858b-8e608fcb2836 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:45.968071 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b7173-076b-49b3-858b-8e608fcb2836 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:46.002317 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd75603d-61f4-4c9c-bdef-4b0f9aac34e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:46.002927 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75603d-61f4-4c9c-bdef-4b0f9aac34e7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:46.003222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75603d-61f4-4c9c-bdef-4b0f9aac34e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:46.003498 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75603d-61f4-4c9c-bdef-4b0f9aac34e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:46.004228 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd75603d-61f4-4c9c-bdef-4b0f9aac34e7 None admin] RBAC: Adding query filter params (role.id=bffbe227cda1418cbb58394ddff6f861, scope.project.id=8e201e5f281a4977be503e22daf149d7, user.id=eb3f6deffb54467f8b923956ed2a6acf) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:46.004530 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd75603d-61f4-4c9c-bdef-4b0f9aac34e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:46.005944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd75603d-61f4-4c9c-bdef-4b0f9aac34e7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:46.019425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 368/722] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:23:45 2022] GET /identity/v3/role_assignments?user.id=eb3f6deffb54467f8b923956ed2a6acf&scope.project.id=8e201e5f281a4977be503e22daf149d7&role.id=bffbe227cda1418cbb58394ddff6f861 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:46.546368 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef866e-0342-490c-8108-2c806924c67a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:46.546866 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef866e-0342-490c-8108-2c806924c67a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:46.546963 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef866e-0342-490c-8108-2c806924c67a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:46.548530 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 355/723] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:46.557256 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841c8e8-288f-44e2-8960-2c2ba0691e46 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:46.557714 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841c8e8-288f-44e2-8960-2c2ba0691e46 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:46.557903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841c8e8-288f-44e2-8960-2c2ba0691e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:46.580483 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0841c8e8-288f-44e2-8960-2c2ba0691e46 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:46.581676 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0841c8e8-288f-44e2-8960-2c2ba0691e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:46.619484 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:46.666848 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 369/724] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:46 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:46.703767 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef866e-0342-490c-8108-2c806924c67a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:46.830669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef866e-0342-490c-8108-2c806924c67a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:46.880450 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110cc0b-deb2-4888-a93e-8893d0a00f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:46.882056 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110cc0b-deb2-4888-a93e-8893d0a00f3f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:46.882620 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110cc0b-deb2-4888-a93e-8893d0a00f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:46.883093 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110cc0b-deb2-4888-a93e-8893d0a00f3f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:46.900022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110cc0b-deb2-4888-a93e-8893d0a00f3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:46.904971 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-4110cc0b-deb2-4888-a93e-8893d0a00f3f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:46.904971 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:46.904971 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:46.904971 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:46.904971 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:46.904971 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:46.904971 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:46.904971 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:46.904971 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:46.904971 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:46.904971 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:46.904971 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:46.904971 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:46.904971 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:46.904971 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:46.904971 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:46.905878 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:46.905878 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:46.905878 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:46.905878 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:46.905878 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:46.905878 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:46.905878 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:46.905878 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:46.905878 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:46.905878 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:46.905878 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:46.905878 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:46.905878 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:46.905878 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:46.905878 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:46.905878 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:46.906722 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:46.906722 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:46.906722 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:46.906722 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:46.906722 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:46.906722 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:46.906722 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:46.906722 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:46.906722 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:46.906722 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:46.906722 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:46.906722 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:46.909008 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 356/725] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 12:23:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:46.915274 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0841c8e8-288f-44e2-8960-2c2ba0691e46 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:47.070509 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0841c8e8-288f-44e2-8960-2c2ba0691e46 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:47.092308 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ce3af-a426-4191-bd5c-11e32ef1c02f None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:47.092929 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ce3af-a426-4191-bd5c-11e32ef1c02f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:47.093405 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ce3af-a426-4191-bd5c-11e32ef1c02f None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:47.094926 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 357/726] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:23:47.099773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c080f-0d17-4dda-8dc9-afca78f480be None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:47.100483 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c080f-0d17-4dda-8dc9-afca78f480be None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:47.100768 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c080f-0d17-4dda-8dc9-afca78f480be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:47.130157 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7d3c080f-0d17-4dda-8dc9-afca78f480be None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:47.132743 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7d3c080f-0d17-4dda-8dc9-afca78f480be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:47.136295 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d472ba3f-f333-41cf-820e-eaa19bef9efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:47.137183 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472ba3f-f333-41cf-820e-eaa19bef9efa None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:47.137663 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472ba3f-f333-41cf-820e-eaa19bef9efa None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:47.138133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472ba3f-f333-41cf-820e-eaa19bef9efa None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:47.138996 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d472ba3f-f333-41cf-820e-eaa19bef9efa None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:47.139518 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d472ba3f-f333-41cf-820e-eaa19bef9efa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:47.141078 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d472ba3f-f333-41cf-820e-eaa19bef9efa None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:47.155284 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 370/727] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 12:23:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:47.163227 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d472ba3f-f333-41cf-820e-eaa19bef9efa None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:47.219523 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:47.239547 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:47.267082 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 358/728] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:47 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:47.317273 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c080f-0d17-4dda-8dc9-afca78f480be None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:47.326701 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d472ba3f-f333-41cf-820e-eaa19bef9efa None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:47.374333 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739141fa-7ae1-41d7-8ec1-5babd1d9f8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:47.374883 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739141fa-7ae1-41d7-8ec1-5babd1d9f8d8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:47.375160 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739141fa-7ae1-41d7-8ec1-5babd1d9f8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:47.376202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739141fa-7ae1-41d7-8ec1-5babd1d9f8d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:47.400982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739141fa-7ae1-41d7-8ec1-5babd1d9f8d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:47.402450 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739141fa-7ae1-41d7-8ec1-5babd1d9f8d8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:47.432128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-739141fa-7ae1-41d7-8ec1-5babd1d9f8d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:47.432128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:47.432128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:47.432128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:47.432128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:47.432128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:47.432128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:47.432128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:47.432128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:47.432128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:47.432128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:47.432128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:47.432128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:47.432128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:47.432128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:47.432128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:47.433164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:47.433164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:47.433164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:47.433164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:47.433164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:47.433164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:47.433164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:47.433164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:47.433164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:47.433164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:47.433164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:47.433164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:47.433164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:47.433164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:47.433164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:47.433164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:47.433164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:47.434161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:47.434161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:47.434161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:47.434737 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 371/729] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:23:47.441390 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739141fa-7ae1-41d7-8ec1-5babd1d9f8d8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:47.525365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c080f-0d17-4dda-8dc9-afca78f480be None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:47.575550 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:47.589285 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e3666-14a2-4f70-8063-5a443897f825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:47.590532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e3666-14a2-4f70-8063-5a443897f825 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:47.596561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e3666-14a2-4f70-8063-5a443897f825 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:47.596561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e3666-14a2-4f70-8063-5a443897f825 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:47.602263 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739141fa-7ae1-41d7-8ec1-5babd1d9f8d8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:47.603791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e3666-14a2-4f70-8063-5a443897f825 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:47.608749 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-727e3666-14a2-4f70-8063-5a443897f825 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:23:47.608749 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:47.608749 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:47.608749 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:47.608749 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:47.608749 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:47.608749 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:47.608749 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:47.608749 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:47.608749 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:47.608749 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:47.608749 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:47.608749 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:47.608749 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:47.608749 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:47.608749 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:47.609726 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:47.609726 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:47.609726 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:47.609726 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:47.609726 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:47.609726 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:47.609726 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:47.609726 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:47.609726 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:47.609726 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:47.609726 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:47.609726 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:47.609726 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:47.609726 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:47.609726 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:47.609726 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:47.610616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:47.610616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:47.610616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:47.610616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:47.610616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:47.610616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:47.610616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:47.610616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:47.610616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:47.610616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:47.610616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:23:47.610616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:47.614309 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 359/730] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:23:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:47.616044 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e35dbd-ba6a-451a-bf42-cce6661cc03f None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:47.616686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e35dbd-ba6a-451a-bf42-cce6661cc03f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:47.617163 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e35dbd-ba6a-451a-bf42-cce6661cc03f None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:47.624017 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 360/731] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:47 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:47.625140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e35dbd-ba6a-451a-bf42-cce6661cc03f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:47.644115 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf08f41-87d3-4735-9d98-675cd59dcf5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:47.644804 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf08f41-87d3-4735-9d98-675cd59dcf5d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:47.645315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf08f41-87d3-4735-9d98-675cd59dcf5d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:47.645594 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf08f41-87d3-4735-9d98-675cd59dcf5d None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:47.646232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf08f41-87d3-4735-9d98-675cd59dcf5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:47.646532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf08f41-87d3-4735-9d98-675cd59dcf5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:47.648900 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf08f41-87d3-4735-9d98-675cd59dcf5d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:47.664965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 372/732] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:47.666634 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b369a-be28-4386-ace0-fcbe24cf7608 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:47.667136 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b369a-be28-4386-ace0-fcbe24cf7608 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:47.667671 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b369a-be28-4386-ace0-fcbe24cf7608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:47.675688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:47.695354 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d08b369a-be28-4386-ace0-fcbe24cf7608 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:47.697666 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d08b369a-be28-4386-ace0-fcbe24cf7608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:47.752022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e35dbd-ba6a-451a-bf42-cce6661cc03f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:47.781358 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 373/733] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:47 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:47.782609 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b369a-be28-4386-ace0-fcbe24cf7608 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:47.805043 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c57aed-fc47-4126-93c5-84d58fd24010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:47.805661 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c57aed-fc47-4126-93c5-84d58fd24010 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:47.805946 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c57aed-fc47-4126-93c5-84d58fd24010 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:47.806221 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c57aed-fc47-4126-93c5-84d58fd24010 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:47.806887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c57aed-fc47-4126-93c5-84d58fd24010 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:47.807223 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c57aed-fc47-4126-93c5-84d58fd24010 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:47.808611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c57aed-fc47-4126-93c5-84d58fd24010 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:47.824455 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 361/734] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:23:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:47.826470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c57aed-fc47-4126-93c5-84d58fd24010 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:47.860217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:47.894388 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b369a-be28-4386-ace0-fcbe24cf7608 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:47.930227 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:47.930815 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b332010-3e00-4083-98b1-b7eebc5834cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:47.931493 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b332010-3e00-4083-98b1-b7eebc5834cd None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:47.931776 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b332010-3e00-4083-98b1-b7eebc5834cd None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:47.932051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b332010-3e00-4083-98b1-b7eebc5834cd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:47.934627 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c57aed-fc47-4126-93c5-84d58fd24010 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:47.939748 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b332010-3e00-4083-98b1-b7eebc5834cd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:47.941262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b332010-3e00-4083-98b1-b7eebc5834cd None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:47.954306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-4b332010-3e00-4083-98b1-b7eebc5834cd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 12:23:47.954306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:47.954306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:47.954306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:47.954306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:47.954306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:47.954306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:47.954306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:47.954306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:47.954306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:47.954306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:47.954306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:47.954306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:23:47.954306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:23:47.954306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:23:47.954306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:23:47.955383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:47.955383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:47.955383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:23:47.955383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:47.955383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:23:47.955383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:47.955383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:47.955383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:47.955383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:47.955383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:47.955383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:47.955383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:47.955383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:47.955383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:47.955383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:47.955383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:47.955383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:47.955383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:47.956319 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:47.956319 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:47.956319 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:23:47.956319 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:23:47.956319 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:23:47.956319 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:23:47.956319 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:23:47.956319 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:23:47.956319 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 12:23:47.956319 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:47.956319 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 374/735] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 10 12:23:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:47.956319 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b332010-3e00-4083-98b1-b7eebc5834cd None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:47.972538 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023bf9f2-5d87-4155-8f8c-0be257079027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:47.973271 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023bf9f2-5d87-4155-8f8c-0be257079027 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:47.973564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023bf9f2-5d87-4155-8f8c-0be257079027 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:47.974055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023bf9f2-5d87-4155-8f8c-0be257079027 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:47.974950 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023bf9f2-5d87-4155-8f8c-0be257079027 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:47.978058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023bf9f2-5d87-4155-8f8c-0be257079027 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:47.987688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-023bf9f2-5d87-4155-8f8c-0be257079027 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 12:23:47.987688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:47.987688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:23:47.987688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:23:47.987688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:23:47.987688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:23:47.987688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:23:47.987688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:23:47.987688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 12:23:47.987688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:47.987688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:23:47.987688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:47.987688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:47.987688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:47.987688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:47.987688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:47.987688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:47.988833 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:47.988833 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:47.988833 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:47.988833 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:47.988833 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:47.988833 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:47.988833 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:23:47.988833 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:23:47.988833 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:23:47.988833 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:23:47.988833 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:47.988833 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:47.988833 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:47.988833 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:47.988833 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:47.988833 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:47.989887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:47.989887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:47.989887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:47.989887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:47.989887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:47.989887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:47.989887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:47.989887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:47.989887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:47.989887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:47.989887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:23:47.989887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:23:47.989887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 12:23:47.989887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:47.989887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 362/736] 10.222.0.49 () {56 vars in 1129 bytes} [Wed Aug 10 12:23:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:23:47.995405 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2557045c-e205-4493-a903-7c51d7e3168f None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:47.995693 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2557045c-e205-4493-a903-7c51d7e3168f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:47.996599 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2557045c-e205-4493-a903-7c51d7e3168f None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:47.998099 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 363/737] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:47 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:47.998719 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2557045c-e205-4493-a903-7c51d7e3168f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:48.083782 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b332010-3e00-4083-98b1-b7eebc5834cd None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:48.106616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2557045c-e205-4493-a903-7c51d7e3168f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:48.114560 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:48.121398 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e70564-29f7-48e4-8fa3-1049ffb1e375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:48.122037 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e70564-29f7-48e4-8fa3-1049ffb1e375 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:48.122326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e70564-29f7-48e4-8fa3-1049ffb1e375 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:48.122598 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e70564-29f7-48e4-8fa3-1049ffb1e375 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:48.134202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e70564-29f7-48e4-8fa3-1049ffb1e375 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:48.135690 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e70564-29f7-48e4-8fa3-1049ffb1e375 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:48.145265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07882ca2-18fe-41aa-8539-80b2d0f05241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:48.145813 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07882ca2-18fe-41aa-8539-80b2d0f05241 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:48.146101 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07882ca2-18fe-41aa-8539-80b2d0f05241 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:48.146380 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07882ca2-18fe-41aa-8539-80b2d0f05241 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:48.147799 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07882ca2-18fe-41aa-8539-80b2d0f05241 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:48.148112 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07882ca2-18fe-41aa-8539-80b2d0f05241 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:48.149533 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07882ca2-18fe-41aa-8539-80b2d0f05241 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:48.151381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-99e70564-29f7-48e4-8fa3-1049ffb1e375 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:48.151381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:48.151381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:48.151381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:48.151381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:48.151381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:48.151381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:48.151381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:48.151381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:48.151381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:48.151381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:48.151381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:48.151381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:48.151381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:48.151381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:48.151381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:48.152417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:48.152417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:48.152417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:48.152417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:48.152417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:48.152417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:48.152417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:48.152417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:48.152417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:48.152417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:48.152417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:48.152417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:48.152417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:48.152417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:48.152417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:48.152417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:48.152417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:48.153292 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:48.153292 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:48.153292 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:48.160907 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 375/738] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:48.161583 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e70564-29f7-48e4-8fa3-1049ffb1e375 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:48.173901 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 364/739] 10.222.0.49 () {56 vars in 1149 bytes} [Wed Aug 10 12:23:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:48.173901 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ee482-a960-4516-8aa2-e427f00057f3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:48.173901 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ee482-a960-4516-8aa2-e427f00057f3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:48.173901 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ee482-a960-4516-8aa2-e427f00057f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:48.189240 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5a5ee482-a960-4516-8aa2-e427f00057f3 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:48.190437 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5a5ee482-a960-4516-8aa2-e427f00057f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:48.206122 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:48.261606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 365/740] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:48 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:23:48.262455 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ee482-a960-4516-8aa2-e427f00057f3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:48.297511 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e70564-29f7-48e4-8fa3-1049ffb1e375 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:48.340129 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b6b1e-17ca-4fb3-9fda-91d5028b38d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:48.340744 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b6b1e-17ca-4fb3-9fda-91d5028b38d6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:48.341023 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b6b1e-17ca-4fb3-9fda-91d5028b38d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:48.341298 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b6b1e-17ca-4fb3-9fda-91d5028b38d6 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:48.342323 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b6b1e-17ca-4fb3-9fda-91d5028b38d6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:48.342643 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b6b1e-17ca-4fb3-9fda-91d5028b38d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:48.344403 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b6b1e-17ca-4fb3-9fda-91d5028b38d6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:48.349073 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 376/741] 10.222.0.49 () {56 vars in 1124 bytes} [Wed Aug 10 12:23:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:48.349699 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b6b1e-17ca-4fb3-9fda-91d5028b38d6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:48.367550 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ee482-a960-4516-8aa2-e427f00057f3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:48.397454 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:48.402273 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270677a-b9e9-4d3a-9907-983ce97e0bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:48.402875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270677a-b9e9-4d3a-9907-983ce97e0bca None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:48.403155 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270677a-b9e9-4d3a-9907-983ce97e0bca None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:48.403454 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270677a-b9e9-4d3a-9907-983ce97e0bca None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:48.404087 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270677a-b9e9-4d3a-9907-983ce97e0bca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:48.404390 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270677a-b9e9-4d3a-9907-983ce97e0bca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:48.405815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270677a-b9e9-4d3a-9907-983ce97e0bca None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:48.417338 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 366/742] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:48.418107 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270677a-b9e9-4d3a-9907-983ce97e0bca None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:48.430428 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:48.452407 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b6b1e-17ca-4fb3-9fda-91d5028b38d6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:48.486501 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdead5f-6fd3-4ff8-979a-a9d64dce3351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:48.487103 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdead5f-6fd3-4ff8-979a-a9d64dce3351 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:48.487405 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdead5f-6fd3-4ff8-979a-a9d64dce3351 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:48.488345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdead5f-6fd3-4ff8-979a-a9d64dce3351 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:48.517270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270677a-b9e9-4d3a-9907-983ce97e0bca None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:48.550428 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddab2dc-17e2-44bd-9ad5-e851c365acd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:48.551025 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddab2dc-17e2-44bd-9ad5-e851c365acd0 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:48.551342 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddab2dc-17e2-44bd-9ad5-e851c365acd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:48.551614 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddab2dc-17e2-44bd-9ad5-e851c365acd0 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:48.552241 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddab2dc-17e2-44bd-9ad5-e851c365acd0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:48.553634 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddab2dc-17e2-44bd-9ad5-e851c365acd0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:48.555338 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdead5f-6fd3-4ff8-979a-a9d64dce3351 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:48.556758 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdead5f-6fd3-4ff8-979a-a9d64dce3351 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:48.602279 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:48.654100 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 377/743] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:23:48 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:23:48.655364 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad6e4f-494a-4f9c-b285-447b934a7619 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:48.655648 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad6e4f-494a-4f9c-b285-447b934a7619 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:48.655915 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad6e4f-494a-4f9c-b285-447b934a7619 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:48.659484 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 378/744] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:48.659484 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad6e4f-494a-4f9c-b285-447b934a7619 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:48.659741 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 367/745] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 10 12:23:48 2022] POST /identity/v3/services => generated 240 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:23:48.659741 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddab2dc-17e2-44bd-9ad5-e851c365acd0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:48.737237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:48.778100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddab2dc-17e2-44bd-9ad5-e851c365acd0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:48.784897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad6e4f-494a-4f9c-b285-447b934a7619 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:48.820937 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86561251-aaa2-44b5-8146-521cc2eb629a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:48.821640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86561251-aaa2-44b5-8146-521cc2eb629a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:48.821933 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86561251-aaa2-44b5-8146-521cc2eb629a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:48.822209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86561251-aaa2-44b5-8146-521cc2eb629a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:48.824914 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445e26a-e9ea-4d16-9f44-07cee684e77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:48.825520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445e26a-e9ea-4d16-9f44-07cee684e77f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:48.825798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445e26a-e9ea-4d16-9f44-07cee684e77f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:48.826070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445e26a-e9ea-4d16-9f44-07cee684e77f None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:48.826698 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445e26a-e9ea-4d16-9f44-07cee684e77f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:48.826995 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445e26a-e9ea-4d16-9f44-07cee684e77f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:48.829155 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445e26a-e9ea-4d16-9f44-07cee684e77f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:48.830374 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86561251-aaa2-44b5-8146-521cc2eb629a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:48.831857 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86561251-aaa2-44b5-8146-521cc2eb629a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:48.833278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 379/746] 10.222.0.49 () {56 vars in 1124 bytes} [Wed Aug 10 12:23:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:48.834072 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445e26a-e9ea-4d16-9f44-07cee684e77f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:48.843996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-86561251-aaa2-44b5-8146-521cc2eb629a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 12:23:48.843996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:48.843996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:48.843996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:48.843996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:48.843996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:48.843996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:48.843996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:48.843996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:48.843996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:48.843996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:48.843996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:48.843996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:23:48.843996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:23:48.843996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:23:48.843996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:23:48.844978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:48.844978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:48.844978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:23:48.844978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:48.844978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:23:48.844978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:48.844978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:48.844978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:48.844978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:48.844978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:48.844978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:48.844978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:48.844978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:48.844978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:48.844978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:48.844978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:48.844978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:48.844978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:48.845917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:48.845917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:48.845917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:23:48.845917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:23:48.845917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:23:48.845917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:23:48.845917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:23:48.845917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:23:48.845917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 12:23:48.845917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:48.845917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 368/747] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:23:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:48.846510 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f504f-2d45-4369-b502-6181383f3fc3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:48.846578 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f504f-2d45-4369-b502-6181383f3fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:48.847154 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f504f-2d45-4369-b502-6181383f3fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:48.854246 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:48.867224 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1b9f504f-2d45-4369-b502-6181383f3fc3 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:48.868439 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1b9f504f-2d45-4369-b502-6181383f3fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:48.941504 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 369/748] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:23:48.942345 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f504f-2d45-4369-b502-6181383f3fc3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:48.958650 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445e26a-e9ea-4d16-9f44-07cee684e77f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:48.998199 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb0d3a-a889-4c1f-b910-bd5d36aa85d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:48.998896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb0d3a-a889-4c1f-b910-bd5d36aa85d3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:48.999203 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb0d3a-a889-4c1f-b910-bd5d36aa85d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:48.999480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb0d3a-a889-4c1f-b910-bd5d36aa85d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:49.003671 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:49.010301 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb0d3a-a889-4c1f-b910-bd5d36aa85d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:49.011787 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb0d3a-a889-4c1f-b910-bd5d36aa85d3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:49.029306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-d6bb0d3a-a889-4c1f-b910-bd5d36aa85d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:49.029306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:49.029306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:49.029306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:49.029306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:49.029306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:49.029306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:49.029306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:49.029306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:49.029306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:49.029306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:49.029306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:49.029306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:49.029306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:49.029306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:49.029306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:49.030565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:49.030565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:49.030565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:49.030565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:49.030565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:49.030565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:49.030565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:49.030565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:49.030565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:49.030565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:49.030565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:49.030565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:49.030565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:49.030565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:49.030565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:49.030565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:49.030565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:49.031626 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:49.031626 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:49.031626 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:49.031626 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 380/749] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:49.031626 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb0d3a-a889-4c1f-b910-bd5d36aa85d3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:49.056805 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f504f-2d45-4369-b502-6181383f3fc3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:49.094146 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1770e5-0add-4ddc-ac9c-c33afaf45562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:49.095003 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1770e5-0add-4ddc-ac9c-c33afaf45562 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:49.095305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1770e5-0add-4ddc-ac9c-c33afaf45562 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:49.095574 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1770e5-0add-4ddc-ac9c-c33afaf45562 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:49.096258 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1770e5-0add-4ddc-ac9c-c33afaf45562 None admin] RBAC: Adding query filter params (interface=public, service_id=f7c38703b9014c87938f905ef8879537, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:49.096566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1770e5-0add-4ddc-ac9c-c33afaf45562 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:49.097920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1770e5-0add-4ddc-ac9c-c33afaf45562 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:49.100996 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:49.105567 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 370/750] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Aug 10 12:23:48 2022] GET /identity/v3/endpoints?service_id=f7c38703b9014c87938f905ef8879537&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:49.106359 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1770e5-0add-4ddc-ac9c-c33afaf45562 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:49.148372 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb0d3a-a889-4c1f-b910-bd5d36aa85d3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:49.185619 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a0381-83eb-4f62-bd88-04db6209e240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:49.186574 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a0381-83eb-4f62-bd88-04db6209e240 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:49.186855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a0381-83eb-4f62-bd88-04db6209e240 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:49.187129 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a0381-83eb-4f62-bd88-04db6209e240 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:49.187885 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a0381-83eb-4f62-bd88-04db6209e240 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:49.188190 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a0381-83eb-4f62-bd88-04db6209e240 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:49.189612 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a0381-83eb-4f62-bd88-04db6209e240 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:49.212286 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 381/751] 10.222.0.49 () {56 vars in 1135 bytes} [Wed Aug 10 12:23:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:49.212286 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a0381-83eb-4f62-bd88-04db6209e240 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:49.220379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1770e5-0add-4ddc-ac9c-c33afaf45562 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:49.241877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:49.259769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd9adc-5cf7-4980-b975-56422560b937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:49.260407 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd9adc-5cf7-4980-b975-56422560b937 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:49.260687 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd9adc-5cf7-4980-b975-56422560b937 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:49.260959 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd9adc-5cf7-4980-b975-56422560b937 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:49.272815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd9adc-5cf7-4980-b975-56422560b937 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:49.278445 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b6bd9adc-5cf7-4980-b975-56422560b937 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:23:49.278445 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:49.278445 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:49.278445 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:49.278445 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:49.278445 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:49.278445 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:49.278445 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:49.278445 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:49.278445 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:49.278445 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:49.278445 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:49.278445 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:49.278445 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:49.278445 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:49.278445 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:49.279621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:49.279621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:49.279621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:49.279621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:49.279621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:49.279621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:49.279621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:49.279621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:49.279621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:49.279621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:49.279621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:49.279621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:49.279621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:49.279621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:49.279621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:49.279621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:49.280475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:49.280475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:49.280475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:49.280475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:49.280475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:49.280475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:49.280475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:49.280475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:49.280475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:49.280475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:49.280475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:23:49.280475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:49.284351 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 371/752] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:23:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:49.285058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd9adc-5cf7-4980-b975-56422560b937 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:49.326763 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a0381-83eb-4f62-bd88-04db6209e240 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:49.348043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:49.363100 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ee5b7-0aad-46ed-b613-36ffe2a0fa2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:49.363744 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ee5b7-0aad-46ed-b613-36ffe2a0fa2e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:49.364024 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ee5b7-0aad-46ed-b613-36ffe2a0fa2e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:49.364322 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ee5b7-0aad-46ed-b613-36ffe2a0fa2e None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:49.364957 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ee5b7-0aad-46ed-b613-36ffe2a0fa2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:49.365262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ee5b7-0aad-46ed-b613-36ffe2a0fa2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:49.367274 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ee5b7-0aad-46ed-b613-36ffe2a0fa2e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:49.383430 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 382/753] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:49.383430 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ee5b7-0aad-46ed-b613-36ffe2a0fa2e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:49.402576 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd9adc-5cf7-4980-b975-56422560b937 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:49.438674 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4ec38-bba4-454f-a640-3fb48a8eaa98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:49.439453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4ec38-bba4-454f-a640-3fb48a8eaa98 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:49.439831 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4ec38-bba4-454f-a640-3fb48a8eaa98 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:49.440189 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4ec38-bba4-454f-a640-3fb48a8eaa98 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:49.440788 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c4ec38-bba4-454f-a640-3fb48a8eaa98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:49.442154 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c4ec38-bba4-454f-a640-3fb48a8eaa98 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:49.455202 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 372/754] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Aug 10 12:23:49 2022] GET /identity/v3/services => generated 772 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:49.455202 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4ec38-bba4-454f-a640-3fb48a8eaa98 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:49.497768 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ee5b7-0aad-46ed-b613-36ffe2a0fa2e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:49.510764 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:49.535962 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8894aca6-6bdc-40f8-a844-2ae67d1f8b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:49.536904 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894aca6-6bdc-40f8-a844-2ae67d1f8b26 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:49.537185 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894aca6-6bdc-40f8-a844-2ae67d1f8b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:49.537460 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894aca6-6bdc-40f8-a844-2ae67d1f8b26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:49.581675 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4ec38-bba4-454f-a640-3fb48a8eaa98 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:49.607374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:49.627548 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b370f-6cf8-4463-bebb-d9e7b553e0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:49.628157 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b370f-6cf8-4463-bebb-d9e7b553e0a8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:49.628458 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b370f-6cf8-4463-bebb-d9e7b553e0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:49.628710 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b370f-6cf8-4463-bebb-d9e7b553e0a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:49.629411 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b370f-6cf8-4463-bebb-d9e7b553e0a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:49.629655 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b370f-6cf8-4463-bebb-d9e7b553e0a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:49.631015 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b370f-6cf8-4463-bebb-d9e7b553e0a8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:49.632951 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8894aca6-6bdc-40f8-a844-2ae67d1f8b26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:49.635129 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8894aca6-6bdc-40f8-a844-2ae67d1f8b26 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:49.641217 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 373/755] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:23:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:49.642357 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b370f-6cf8-4463-bebb-d9e7b553e0a8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:49.762072 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 383/756] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:23:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:23:49.763917 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8894aca6-6bdc-40f8-a844-2ae67d1f8b26 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:49.773239 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b370f-6cf8-4463-bebb-d9e7b553e0a8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:49.781824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:49.812807 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a0a31-341d-4cc2-8efc-a53e85a47988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:49.813468 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a0a31-341d-4cc2-8efc-a53e85a47988 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:49.813747 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a0a31-341d-4cc2-8efc-a53e85a47988 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:49.814029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a0a31-341d-4cc2-8efc-a53e85a47988 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:49.822452 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a0a31-341d-4cc2-8efc-a53e85a47988 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:49.823964 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a0a31-341d-4cc2-8efc-a53e85a47988 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:49.836554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-ee4a0a31-341d-4cc2-8efc-a53e85a47988 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:49.836554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:49.836554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:49.836554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:49.836554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:49.836554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:49.836554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:49.836554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:49.836554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:49.836554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:49.836554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:49.836554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:49.836554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:23:49.836554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:23:49.836554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:23:49.837696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:23:49.837696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:49.837696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:49.837696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:49.837696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:49.837696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:49.837696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:49.837696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:49.837696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:49.837696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:49.837696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:49.837696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:49.837696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:49.837696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:49.837696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:49.837696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:49.837696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:49.838801 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:23:49.838801 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:23:49.838801 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:23:49.838801 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:23:49.838801 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:23:49.838801 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:23:49.838801 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:49.838801 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:49.842583 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 374/757] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:49.843297 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a0a31-341d-4cc2-8efc-a53e85a47988 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:49.869792 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:49.896175 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8894aca6-6bdc-40f8-a844-2ae67d1f8b26 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:49.942536 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb1dc8-f4ed-4cba-b871-a8b5b71b7b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:49.943682 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb1dc8-f4ed-4cba-b871-a8b5b71b7b6c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:49.944214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb1dc8-f4ed-4cba-b871-a8b5b71b7b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:49.944983 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb1dc8-f4ed-4cba-b871-a8b5b71b7b6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:49.960491 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cb1dc8-f4ed-4cba-b871-a8b5b71b7b6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:49.962212 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cb1dc8-f4ed-4cba-b871-a8b5b71b7b6c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:49.966627 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a0a31-341d-4cc2-8efc-a53e85a47988 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:49.979852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-23cb1dc8-f4ed-4cba-b871-a8b5b71b7b6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:49.979852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:49.979852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:49.979852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:49.979852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:49.979852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:49.979852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:49.979852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:49.979852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:49.979852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:49.979852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:49.979852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:49.979852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:49.979852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:49.979852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:49.979852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:49.980846 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:49.980846 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:49.980846 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:49.980846 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:49.980846 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:49.980846 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:49.980846 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:49.980846 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:49.980846 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:49.980846 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:49.980846 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:49.980846 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:49.980846 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:49.980846 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:49.980846 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:49.980846 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:49.980846 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:49.981811 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:49.981811 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:49.981811 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:49.985736 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 384/758] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:49.986573 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb1dc8-f4ed-4cba-b871-a8b5b71b7b6c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:50.006940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04690cd-0f53-4b41-a0d6-2ce93b7278af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:50.008013 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04690cd-0f53-4b41-a0d6-2ce93b7278af None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:50.008541 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04690cd-0f53-4b41-a0d6-2ce93b7278af None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:50.009035 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04690cd-0f53-4b41-a0d6-2ce93b7278af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:50.020661 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04690cd-0f53-4b41-a0d6-2ce93b7278af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:50.022971 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04690cd-0f53-4b41-a0d6-2ce93b7278af None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:50.038457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a04690cd-0f53-4b41-a0d6-2ce93b7278af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:50.038457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:50.038457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:50.038457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:50.038457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:50.038457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:50.038457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:50.038457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:50.038457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:50.038457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:50.038457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:50.038457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:50.038457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:50.038457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:50.038457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:50.038457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:50.039459 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:50.039459 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:50.039459 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:50.039459 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:50.039459 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:50.039459 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:50.039459 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:50.039459 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:50.039459 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:50.039459 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:50.039459 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:50.039459 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:50.039459 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:50.039459 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:50.039459 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:50.039459 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:50.039459 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:50.040336 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:50.040336 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:50.040336 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:50.042364 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 375/759] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:50.043414 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04690cd-0f53-4b41-a0d6-2ce93b7278af None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:50.071667 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:50.097253 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb1dc8-f4ed-4cba-b871-a8b5b71b7b6c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:50.127327 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:50.133162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8253c5-d52d-4c96-a2ce-cc958ac4234f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:50.133830 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8253c5-d52d-4c96-a2ce-cc958ac4234f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:50.134051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8253c5-d52d-4c96-a2ce-cc958ac4234f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:50.134329 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8253c5-d52d-4c96-a2ce-cc958ac4234f None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:50.135027 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8253c5-d52d-4c96-a2ce-cc958ac4234f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:50.135391 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8253c5-d52d-4c96-a2ce-cc958ac4234f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:50.136924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8253c5-d52d-4c96-a2ce-cc958ac4234f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:50.153731 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04690cd-0f53-4b41-a0d6-2ce93b7278af None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:50.155315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 385/760] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:23:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:50.156260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8253c5-d52d-4c96-a2ce-cc958ac4234f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:50.192260 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7b7da-8bcc-4d04-a8fb-93591f379adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:50.193276 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7b7da-8bcc-4d04-a8fb-93591f379adc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:50.193734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7b7da-8bcc-4d04-a8fb-93591f379adc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:50.194510 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7b7da-8bcc-4d04-a8fb-93591f379adc None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:50.195385 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7b7da-8bcc-4d04-a8fb-93591f379adc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:50.195887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7b7da-8bcc-4d04-a8fb-93591f379adc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:50.197434 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7b7da-8bcc-4d04-a8fb-93591f379adc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:50.210617 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 376/761] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:50.211806 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7b7da-8bcc-4d04-a8fb-93591f379adc None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:50.278163 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8253c5-d52d-4c96-a2ce-cc958ac4234f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:50.303607 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:50.315725 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef5c0f-0d7d-4f88-8d76-335b70b5aa31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:50.316346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef5c0f-0d7d-4f88-8d76-335b70b5aa31 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:50.316667 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef5c0f-0d7d-4f88-8d76-335b70b5aa31 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:50.316986 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef5c0f-0d7d-4f88-8d76-335b70b5aa31 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:50.317627 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef5c0f-0d7d-4f88-8d76-335b70b5aa31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:50.317934 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef5c0f-0d7d-4f88-8d76-335b70b5aa31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:50.319930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef5c0f-0d7d-4f88-8d76-335b70b5aa31 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:50.328560 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7b7da-8bcc-4d04-a8fb-93591f379adc None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:50.335553 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 386/762] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:50.336714 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef5c0f-0d7d-4f88-8d76-335b70b5aa31 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:50.365932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d8561-6b66-4229-aac9-c135385af5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:50.366919 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d8561-6b66-4229-aac9-c135385af5de None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:50.367499 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d8561-6b66-4229-aac9-c135385af5de None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:50.367974 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d8561-6b66-4229-aac9-c135385af5de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:50.369125 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d8561-6b66-4229-aac9-c135385af5de None admin] RBAC: Adding query filter params (role.id=a9bbb011afca4176a3def9356cb5f165, scope.project.id=8e201e5f281a4977be503e22daf149d7, user.id=e42059e9af9842548ea08dcd39ff0b6e) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:50.369609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d8561-6b66-4229-aac9-c135385af5de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:50.371681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d8561-6b66-4229-aac9-c135385af5de None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:50.378097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 377/763] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:23:50 2022] GET /identity/v3/role_assignments?user.id=e42059e9af9842548ea08dcd39ff0b6e&scope.project.id=8e201e5f281a4977be503e22daf149d7&role.id=a9bbb011afca4176a3def9356cb5f165 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:50.379169 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d8561-6b66-4229-aac9-c135385af5de None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:50.408350 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:50.470985 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef5c0f-0d7d-4f88-8d76-335b70b5aa31 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:50.494139 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d8561-6b66-4229-aac9-c135385af5de None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:50.510644 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088d442-80b4-4af1-aa3f-eae09810dc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:50.511265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088d442-80b4-4af1-aa3f-eae09810dc4c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:50.511548 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088d442-80b4-4af1-aa3f-eae09810dc4c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:50.511825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088d442-80b4-4af1-aa3f-eae09810dc4c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:50.520198 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e088d442-80b4-4af1-aa3f-eae09810dc4c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:50.522013 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e088d442-80b4-4af1-aa3f-eae09810dc4c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:50.533803 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719df60b-5098-4bf2-93d6-c570bc7f0d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:50.534766 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719df60b-5098-4bf2-93d6-c570bc7f0d97 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:50.535278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-e088d442-80b4-4af1-aa3f-eae09810dc4c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:23:50.535278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:50.535278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:50.535278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:50.535278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:50.535278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:50.535278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:50.535278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:50.535278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:50.535278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:50.535278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:50.535278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:50.535278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:23:50.535278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:23:50.535278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:23:50.535278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:23:50.536184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:50.536184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:50.536184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:23:50.536184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:50.536184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:23:50.536184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:50.536184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:50.536184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:50.536184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:50.536184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:50.536184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:50.536184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:50.536184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:50.536184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:50.536184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:50.536184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:50.536184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:50.536184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:50.537500 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:50.537500 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:50.537500 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:23:50.537500 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:23:50.537500 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:23:50.537500 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:23:50.537500 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:23:50.537500 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:23:50.537500 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:23:50.537500 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:50.538860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719df60b-5098-4bf2-93d6-c570bc7f0d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:50.538943 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 387/764] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Aug 10 12:23:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:50.539510 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719df60b-5098-4bf2-93d6-c570bc7f0d97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:50.543117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088d442-80b4-4af1-aa3f-eae09810dc4c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:50.549449 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719df60b-5098-4bf2-93d6-c570bc7f0d97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:50.551260 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719df60b-5098-4bf2-93d6-c570bc7f0d97 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:50.564905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-719df60b-5098-4bf2-93d6-c570bc7f0d97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:50.564905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:50.564905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:50.564905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:50.564905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:50.564905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:50.564905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:50.564905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:50.564905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:50.564905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:50.564905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:50.564905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:50.564905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:23:50.564905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:23:50.564905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:23:50.565845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:23:50.565845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:50.565845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:50.565845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:50.565845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:50.565845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:50.565845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:50.565845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:50.565845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:50.565845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:50.565845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:50.565845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:50.565845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:50.565845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:50.565845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:50.565845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:50.565845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:50.566725 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:23:50.566725 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:23:50.566725 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:23:50.566725 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:23:50.566725 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:23:50.566725 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:23:50.566725 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:50.566725 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:50.569164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 378/765] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:50.577683 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719df60b-5098-4bf2-93d6-c570bc7f0d97 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:50.581861 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:50.747163 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088d442-80b4-4af1-aa3f-eae09810dc4c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:50.770513 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719df60b-5098-4bf2-93d6-c570bc7f0d97 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:50.776304 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:50.813362 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3482271f-7778-4933-90a1-acb4899b4c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:50.813910 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482271f-7778-4933-90a1-acb4899b4c52 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:50.814197 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482271f-7778-4933-90a1-acb4899b4c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:50.814477 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482271f-7778-4933-90a1-acb4899b4c52 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:50.815231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3482271f-7778-4933-90a1-acb4899b4c52 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:50.815548 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3482271f-7778-4933-90a1-acb4899b4c52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:50.816985 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3482271f-7778-4933-90a1-acb4899b4c52 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:50.842601 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 388/766] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Aug 10 12:23:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:23:50.843899 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14dfe15-ea93-4ad5-ac4c-2fa1e1374240 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:50.844186 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14dfe15-ea93-4ad5-ac4c-2fa1e1374240 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:50.844461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14dfe15-ea93-4ad5-ac4c-2fa1e1374240 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:50.845115 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b558a-bfba-4e5c-a5ca-5713bed615d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:50.845792 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b558a-bfba-4e5c-a5ca-5713bed615d1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:50.845897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 389/767] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:50.847543 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b558a-bfba-4e5c-a5ca-5713bed615d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:50.847891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b558a-bfba-4e5c-a5ca-5713bed615d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:50.848677 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b558a-bfba-4e5c-a5ca-5713bed615d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:50.850297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14dfe15-ea93-4ad5-ac4c-2fa1e1374240 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:50.850654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b558a-bfba-4e5c-a5ca-5713bed615d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:50.852131 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b558a-bfba-4e5c-a5ca-5713bed615d1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:50.865337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 379/768] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:23:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:50.867504 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562646d-bdfe-42e2-9acf-56bc2927814f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:50.867792 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562646d-bdfe-42e2-9acf-56bc2927814f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:50.868069 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562646d-bdfe-42e2-9acf-56bc2927814f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:50.888038 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d562646d-bdfe-42e2-9acf-56bc2927814f None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:50.889246 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d562646d-bdfe-42e2-9acf-56bc2927814f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:50.964114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14dfe15-ea93-4ad5-ac4c-2fa1e1374240 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:50.972214 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:50.990696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 380/769] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:23:50.992271 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562646d-bdfe-42e2-9acf-56bc2927814f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:51.014265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca04214-d275-4510-a924-bf6427f74781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:51.019295 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca04214-d275-4510-a924-bf6427f74781 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:51.019295 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca04214-d275-4510-a924-bf6427f74781 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:51.019295 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca04214-d275-4510-a924-bf6427f74781 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:51.034354 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca04214-d275-4510-a924-bf6427f74781 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:51.036400 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca04214-d275-4510-a924-bf6427f74781 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:51.053796 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-fca04214-d275-4510-a924-bf6427f74781 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:51.053796 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:51.053796 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:51.053796 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:51.053796 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:51.053796 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:51.053796 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:51.053796 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:51.053796 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:51.053796 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:51.053796 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:51.053796 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:51.053796 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:51.053796 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:51.053796 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:51.053796 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:51.054988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:51.054988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:51.054988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:51.054988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:51.054988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:51.054988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:51.054988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:51.054988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:51.054988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:51.054988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:51.054988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:51.054988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:51.054988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:51.054988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:51.054988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:51.054988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:51.054988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:51.065114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:51.065114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:51.065114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:51.065114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 390/770] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:51.065114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca04214-d275-4510-a924-bf6427f74781 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:51.091661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:51.123474 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562646d-bdfe-42e2-9acf-56bc2927814f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:51.162698 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29868da3-fa50-487b-9628-c2a4f0f86be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:51.163684 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29868da3-fa50-487b-9628-c2a4f0f86be1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:51.164277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29868da3-fa50-487b-9628-c2a4f0f86be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:51.165047 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29868da3-fa50-487b-9628-c2a4f0f86be1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:51.166021 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29868da3-fa50-487b-9628-c2a4f0f86be1 None admin] RBAC: Adding query filter params (role.id=bffbe227cda1418cbb58394ddff6f861, scope.project.id=8e201e5f281a4977be503e22daf149d7, user.id=59395bfa505c4540b3957deee78eb698) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:51.166513 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29868da3-fa50-487b-9628-c2a4f0f86be1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:51.168687 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29868da3-fa50-487b-9628-c2a4f0f86be1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:51.183072 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 381/771] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:23:50 2022] GET /identity/v3/role_assignments?user.id=59395bfa505c4540b3957deee78eb698&scope.project.id=8e201e5f281a4977be503e22daf149d7&role.id=bffbe227cda1418cbb58394ddff6f861 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:51.183072 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29868da3-fa50-487b-9628-c2a4f0f86be1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:51.193317 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca04214-d275-4510-a924-bf6427f74781 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:51.251605 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:51.298711 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29868da3-fa50-487b-9628-c2a4f0f86be1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:51.337200 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e5e44-f8e2-4ca2-9fd3-a1741ac4f82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:51.337742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e5e44-f8e2-4ca2-9fd3-a1741ac4f82d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:51.338025 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e5e44-f8e2-4ca2-9fd3-a1741ac4f82d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:51.338298 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e5e44-f8e2-4ca2-9fd3-a1741ac4f82d None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:51.338958 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e5e44-f8e2-4ca2-9fd3-a1741ac4f82d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:51.339296 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e5e44-f8e2-4ca2-9fd3-a1741ac4f82d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:51.340807 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e5e44-f8e2-4ca2-9fd3-a1741ac4f82d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:51.341632 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514bc33-e502-4174-bfc1-2acb6f17737e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:51.342458 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514bc33-e502-4174-bfc1-2acb6f17737e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:51.342886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514bc33-e502-4174-bfc1-2acb6f17737e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:51.343344 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514bc33-e502-4174-bfc1-2acb6f17737e None admin] PATH_INFO: `/v3/services/network` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:51.344073 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514bc33-e502-4174-bfc1-2acb6f17737e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:51.345594 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514bc33-e502-4174-bfc1-2acb6f17737e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:51.355219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 382/772] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:51.361788 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e5e44-f8e2-4ca2-9fd3-a1741ac4f82d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:51.364807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-3514bc33-e502-4174-bfc1-2acb6f17737e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 12:23:51.364807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:51.364807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:23:51.364807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:23:51.364807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:23:51.364807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:23:51.364807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:23:51.364807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:23:51.364807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 12:23:51.364807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:51.364807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:23:51.364807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:51.364807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:51.364807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:51.364807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:51.364807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:51.364807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:51.366340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:51.366340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:51.366340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:51.366340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:51.366340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:51.366340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:51.366340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:23:51.366340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:23:51.366340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:23:51.366340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:23:51.366340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:51.366340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:51.366340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:51.366340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:51.366340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:51.366340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:51.367401 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:51.367401 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:51.367401 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:51.367401 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:51.367401 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:51.367401 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:51.367401 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:51.367401 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:51.367401 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:51.367401 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:51.367401 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:23:51.367401 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:23:51.367401 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 12:23:51.367401 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:51.370507 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 391/773] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:51 2022] GET /identity/v3/services/network => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:51.376878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514bc33-e502-4174-bfc1-2acb6f17737e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:51.528261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:51.550125 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e5e44-f8e2-4ca2-9fd3-a1741ac4f82d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:51.551031 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514bc33-e502-4174-bfc1-2acb6f17737e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:51.575546 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:51.586747 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30393232-a704-4b68-949e-d809a8001093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:51.587868 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30393232-a704-4b68-949e-d809a8001093 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:51.588109 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30393232-a704-4b68-949e-d809a8001093 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:51.588385 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30393232-a704-4b68-949e-d809a8001093 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:51.589015 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30393232-a704-4b68-949e-d809a8001093 None admin] RBAC: Adding query filter params (name=network) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:51.589320 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30393232-a704-4b68-949e-d809a8001093 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:51.590684 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30393232-a704-4b68-949e-d809a8001093 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:51.594477 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac31ded-e610-412d-b2f2-229e672bf86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:51.598139 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac31ded-e610-412d-b2f2-229e672bf86e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:51.598139 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac31ded-e610-412d-b2f2-229e672bf86e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:51.598139 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac31ded-e610-412d-b2f2-229e672bf86e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:51.598601 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 392/774] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 10 12:23:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:51.598601 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd0f93-03c1-4138-8d37-b730c901407f None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:51.598601 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd0f93-03c1-4138-8d37-b730c901407f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:51.598601 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd0f93-03c1-4138-8d37-b730c901407f None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:51.599646 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 393/775] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:51.602963 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac31ded-e610-412d-b2f2-229e672bf86e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:51.603113 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdd0f93-03c1-4138-8d37-b730c901407f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:51.604481 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac31ded-e610-412d-b2f2-229e672bf86e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:51.617350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-2ac31ded-e610-412d-b2f2-229e672bf86e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:51.617350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:51.617350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:51.617350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:51.617350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:51.617350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:51.617350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:51.617350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:51.617350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:51.617350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:51.617350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:51.617350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:51.617350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:23:51.617350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:23:51.617350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:23:51.619314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:23:51.619314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:51.619314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:51.619314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:51.619314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:51.619314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:51.619314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:51.619314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:51.619314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:51.619314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:51.619314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:51.619314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:51.619314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:51.619314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:51.619314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:51.619314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:51.619314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:51.621740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:23:51.621740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:23:51.621740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:23:51.621740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:23:51.621740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:23:51.621740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:23:51.621740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:51.621740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:51.621740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 383/776] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:23:51.621740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caded356-4403-4487-8e4b-bc2fe0c88ec8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:51.621740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caded356-4403-4487-8e4b-bc2fe0c88ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:51.623316 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caded356-4403-4487-8e4b-bc2fe0c88ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:51.641957 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-caded356-4403-4487-8e4b-bc2fe0c88ec8 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:51.643333 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-caded356-4403-4487-8e4b-bc2fe0c88ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:51.719213 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 384/777] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:23:51.719213 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caded356-4403-4487-8e4b-bc2fe0c88ec8 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:51.734881 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdd0f93-03c1-4138-8d37-b730c901407f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:51.786056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd5e57-95c4-4cad-8f93-7bc2ffd612a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:51.786762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd5e57-95c4-4cad-8f93-7bc2ffd612a2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:51.787153 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd5e57-95c4-4cad-8f93-7bc2ffd612a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:51.787660 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd5e57-95c4-4cad-8f93-7bc2ffd612a2 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:51.788414 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd5e57-95c4-4cad-8f93-7bc2ffd612a2 None admin] RBAC: Adding query filter params (type=network) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:51.788790 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd5e57-95c4-4cad-8f93-7bc2ffd612a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:51.790295 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd5e57-95c4-4cad-8f93-7bc2ffd612a2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:51.800057 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 394/778] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 10 12:23:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:51.800057 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd5e57-95c4-4cad-8f93-7bc2ffd612a2 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:51.827637 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:51.846022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caded356-4403-4487-8e4b-bc2fe0c88ec8 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:51.854876 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:51.882640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647089b-f7ac-4fb0-b48c-b146703af8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:51.883509 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647089b-f7ac-4fb0-b48c-b146703af8c8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:51.884627 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647089b-f7ac-4fb0-b48c-b146703af8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:51.884627 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647089b-f7ac-4fb0-b48c-b146703af8c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:51.885262 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647089b-f7ac-4fb0-b48c-b146703af8c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:51.885798 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647089b-f7ac-4fb0-b48c-b146703af8c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:51.887404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647089b-f7ac-4fb0-b48c-b146703af8c8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:51.899510 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 385/779] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:23:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:51.900852 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647089b-f7ac-4fb0-b48c-b146703af8c8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:51.906620 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd5e57-95c4-4cad-8f93-7bc2ffd612a2 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:51.945427 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96622e-8c53-44d8-8f4e-3bf650bda94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:51.946132 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96622e-8c53-44d8-8f4e-3bf650bda94f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:51.946417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96622e-8c53-44d8-8f4e-3bf650bda94f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:51.946696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96622e-8c53-44d8-8f4e-3bf650bda94f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:51.947357 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96622e-8c53-44d8-8f4e-3bf650bda94f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:51.948846 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96622e-8c53-44d8-8f4e-3bf650bda94f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:51.958296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-2e96622e-8c53-44d8-8f4e-3bf650bda94f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 12:23:51.958296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:51.958296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:23:51.958296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:23:51.958296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:23:51.958296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:23:51.958296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:23:51.958296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:23:51.958296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 12:23:51.958296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:51.958296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:23:51.958296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:51.958296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:51.958296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:51.958296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:51.958296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:51.958296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:51.959310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:51.959310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:51.959310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:51.959310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:51.959310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:51.959310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:51.959310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:23:51.959310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:23:51.959310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:23:51.959310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:23:51.959310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:51.959310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:51.959310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:51.959310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:51.959310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:51.959310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:51.960307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:51.960307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:51.960307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:51.960307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:51.960307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:51.960307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:51.960307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:51.960307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:51.960307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:51.960307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:51.960307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:23:51.960307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:23:51.960307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 12:23:51.960307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:51.960307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 395/780] 10.222.0.49 () {56 vars in 1129 bytes} [Wed Aug 10 12:23:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:51.961083 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96622e-8c53-44d8-8f4e-3bf650bda94f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:52.015504 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647089b-f7ac-4fb0-b48c-b146703af8c8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:52.050848 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57522b8-8f91-4be0-9e94-5afcd64cb816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:52.051838 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57522b8-8f91-4be0-9e94-5afcd64cb816 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:52.052318 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57522b8-8f91-4be0-9e94-5afcd64cb816 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:52.052783 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57522b8-8f91-4be0-9e94-5afcd64cb816 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:52.053800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57522b8-8f91-4be0-9e94-5afcd64cb816 None admin] RBAC: Adding query filter params (interface=public, service_id=a691057a64864cceaf975f1f53b3885a, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:52.054398 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57522b8-8f91-4be0-9e94-5afcd64cb816 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:52.055637 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:52.056242 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57522b8-8f91-4be0-9e94-5afcd64cb816 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:52.066096 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 386/781] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Aug 10 12:23:51 2022] GET /identity/v3/endpoints?service_id=a691057a64864cceaf975f1f53b3885a&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:52.067220 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57522b8-8f91-4be0-9e94-5afcd64cb816 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:52.071891 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96622e-8c53-44d8-8f4e-3bf650bda94f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:52.099514 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:52.110445 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91731ca2-3472-44ad-895a-43ff184f5ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:52.111019 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91731ca2-3472-44ad-895a-43ff184f5ee9 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:52.111441 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91731ca2-3472-44ad-895a-43ff184f5ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:52.111818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91731ca2-3472-44ad-895a-43ff184f5ee9 None admin] PATH_INFO: `/v3/projects/8e201e5f281a4977be503e22daf149d7/users/eb3f6deffb54467f8b923956ed2a6acf/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:52.114197 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91731ca2-3472-44ad-895a-43ff184f5ee9 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e201e5f281a4977be503e22daf149d7, user_id=eb3f6deffb54467f8b923956ed2a6acf, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:52.116383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91731ca2-3472-44ad-895a-43ff184f5ee9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:52.172659 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57522b8-8f91-4be0-9e94-5afcd64cb816 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:52.198080 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 396/782] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 12:23:51 2022] PUT /identity/v3/projects/8e201e5f281a4977be503e22daf149d7/users/eb3f6deffb54467f8b923956ed2a6acf/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:23:52.199906 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91731ca2-3472-44ad-895a-43ff184f5ee9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:52.225955 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b10ec8-e194-4bfa-ab1b-8ce6e078865d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:52.227964 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b10ec8-e194-4bfa-ab1b-8ce6e078865d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:52.229094 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b10ec8-e194-4bfa-ab1b-8ce6e078865d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:52.229567 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b10ec8-e194-4bfa-ab1b-8ce6e078865d None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:52.231650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b10ec8-e194-4bfa-ab1b-8ce6e078865d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:52.232181 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b10ec8-e194-4bfa-ab1b-8ce6e078865d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:52.235049 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b10ec8-e194-4bfa-ab1b-8ce6e078865d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:52.247101 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 387/783] 10.222.0.49 () {56 vars in 1124 bytes} [Wed Aug 10 12:23:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:52.253480 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b10ec8-e194-4bfa-ab1b-8ce6e078865d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:52.390757 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:52.394234 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91731ca2-3472-44ad-895a-43ff184f5ee9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:52.400237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b10ec8-e194-4bfa-ab1b-8ce6e078865d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:52.437823 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c81dd-530b-4ee6-9773-275ef5849403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:52.438456 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c81dd-530b-4ee6-9773-275ef5849403 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:52.438741 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c81dd-530b-4ee6-9773-275ef5849403 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:52.439018 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c81dd-530b-4ee6-9773-275ef5849403 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:52.439809 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c81dd-530b-4ee6-9773-275ef5849403 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:52.441791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c81dd-530b-4ee6-9773-275ef5849403 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:52.447460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d75fc4-2138-42b0-adef-1783dde2a872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:52.448399 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d75fc4-2138-42b0-adef-1783dde2a872 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:52.448884 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d75fc4-2138-42b0-adef-1783dde2a872 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:52.449272 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 397/784] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Aug 10 12:23:52 2022] GET /identity/v3/services => generated 772 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:52.449537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d75fc4-2138-42b0-adef-1783dde2a872 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:52.450178 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d75fc4-2138-42b0-adef-1783dde2a872 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:52.450490 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d75fc4-2138-42b0-adef-1783dde2a872 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:52.452709 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d75fc4-2138-42b0-adef-1783dde2a872 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:52.461764 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 388/785] 10.222.0.49 () {56 vars in 1124 bytes} [Wed Aug 10 12:23:52 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:23:52.467293 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c81dd-530b-4ee6-9773-275ef5849403 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:52.474007 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:52.589759 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c81dd-530b-4ee6-9773-275ef5849403 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:52.627815 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713efafb-0132-405c-8f15-4256cbec617e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:52.628437 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713efafb-0132-405c-8f15-4256cbec617e None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:52.628719 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713efafb-0132-405c-8f15-4256cbec617e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:52.628995 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713efafb-0132-405c-8f15-4256cbec617e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:52.629666 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713efafb-0132-405c-8f15-4256cbec617e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:52.631034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713efafb-0132-405c-8f15-4256cbec617e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:52.725998 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 398/786] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:23:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:23:52.838552 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d5ed6-642b-4d96-bc21-4497175ae7a1 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:52.839267 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d5ed6-642b-4d96-bc21-4497175ae7a1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:52.839776 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d5ed6-642b-4d96-bc21-4497175ae7a1 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:52.841679 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 389/787] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:52.847102 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8aa9a-3384-42b4-94b1-d1eb9fd0d134 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:52.847645 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8aa9a-3384-42b4-94b1-d1eb9fd0d134 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:52.847721 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8aa9a-3384-42b4-94b1-d1eb9fd0d134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:52.873171 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-8ee8aa9a-3384-42b4-94b1-d1eb9fd0d134 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:52.874449 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8ee8aa9a-3384-42b4-94b1-d1eb9fd0d134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:52.941619 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:52.961388 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 399/788] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:23:52.988637 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d5ed6-642b-4d96-bc21-4497175ae7a1 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:53.100975 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d5ed6-642b-4d96-bc21-4497175ae7a1 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:53.138596 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb48329-9d66-47c3-ab63-05f9fc7b2845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:53.139310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb48329-9d66-47c3-ab63-05f9fc7b2845 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:53.139587 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb48329-9d66-47c3-ab63-05f9fc7b2845 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:53.139895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb48329-9d66-47c3-ab63-05f9fc7b2845 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:53.150572 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb48329-9d66-47c3-ab63-05f9fc7b2845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:53.155151 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-2cb48329-9d66-47c3-ab63-05f9fc7b2845 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:53.155151 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:53.155151 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:53.155151 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:53.155151 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:53.155151 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:53.155151 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:53.155151 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:53.155151 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:53.155151 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:53.155151 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:53.155151 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:53.155151 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:53.155151 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:53.155151 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:53.155151 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:53.156079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:53.156079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:53.156079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:53.156079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:53.156079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:53.156079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:53.156079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:53.156079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:53.156079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:53.156079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:53.156079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:53.156079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:53.156079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:53.156079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:53.156079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:53.156079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:53.156922 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:53.156922 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:53.156922 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:53.156922 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:53.156922 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:53.156922 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:53.156922 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:53.156922 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:53.156922 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:53.156922 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:53.156922 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:53.156922 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:53.156922 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 390/789] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 12:23:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:53.162145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8aa9a-3384-42b4-94b1-d1eb9fd0d134 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:53.274196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8aa9a-3384-42b4-94b1-d1eb9fd0d134 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:53.324934 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11534430-e290-43f7-b711-8f79940bad5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:53.325551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11534430-e290-43f7-b711-8f79940bad5a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:53.325829 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11534430-e290-43f7-b711-8f79940bad5a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:53.326101 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11534430-e290-43f7-b711-8f79940bad5a None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:53.326768 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11534430-e290-43f7-b711-8f79940bad5a None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:53.327072 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11534430-e290-43f7-b711-8f79940bad5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:53.328463 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11534430-e290-43f7-b711-8f79940bad5a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:53.337408 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 400/790] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 12:23:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:53.342974 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb48329-9d66-47c3-ab63-05f9fc7b2845 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:53.372640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:53.490330 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb48329-9d66-47c3-ab63-05f9fc7b2845 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:53.530943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f28d12-e4f0-4962-8a4a-737c1b695bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:53.531523 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f28d12-e4f0-4962-8a4a-737c1b695bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:53.531798 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f28d12-e4f0-4962-8a4a-737c1b695bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:53.532064 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f28d12-e4f0-4962-8a4a-737c1b695bf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:53.546836 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f28d12-e4f0-4962-8a4a-737c1b695bf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:53.548323 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f28d12-e4f0-4962-8a4a-737c1b695bf4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:53.561587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48297b9-8cf7-415c-bc4b-fc0751cefc52 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:53.562194 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48297b9-8cf7-415c-bc4b-fc0751cefc52 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:53.562670 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48297b9-8cf7-415c-bc4b-fc0751cefc52 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:53.564523 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 401/791] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:23:53.567744 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-e9f28d12-e4f0-4962-8a4a-737c1b695bf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:53.567744 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:53.567744 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:53.567744 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:53.567744 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:53.567744 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:53.567744 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:53.567744 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:53.567744 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:53.567744 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:53.567744 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:53.567744 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:53.567744 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:53.567744 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:53.567744 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:53.567744 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:53.568742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:53.568742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:53.568742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:53.568742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:53.568742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:53.568742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:53.568742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:53.568742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:53.568742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:53.568742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:53.568742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:53.568742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:53.568742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:53.568742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:53.568742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:53.568742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:53.568742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:53.569919 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:53.569919 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:53.569919 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:53.571460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 391/792] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:53.572978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c01c8-6d32-4b08-ad48-fef326e710f3 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:53.573718 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c01c8-6d32-4b08-ad48-fef326e710f3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:53.574194 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c01c8-6d32-4b08-ad48-fef326e710f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:53.574364 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f28d12-e4f0-4962-8a4a-737c1b695bf4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:53.592964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-db4c01c8-6d32-4b08-ad48-fef326e710f3 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:53.594495 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-db4c01c8-6d32-4b08-ad48-fef326e710f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:53.648253 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:53.702139 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:53.709364 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 402/793] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:23:53.733418 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c01c8-6d32-4b08-ad48-fef326e710f3 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:53.769304 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f28d12-e4f0-4962-8a4a-737c1b695bf4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:53.835226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a213896-d219-4f2f-a2d4-48049be39235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:53.835845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a213896-d219-4f2f-a2d4-48049be39235 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:53.836132 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a213896-d219-4f2f-a2d4-48049be39235 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:53.836405 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a213896-d219-4f2f-a2d4-48049be39235 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:53.837048 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a213896-d219-4f2f-a2d4-48049be39235 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:53.837358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a213896-d219-4f2f-a2d4-48049be39235 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:53.838725 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a213896-d219-4f2f-a2d4-48049be39235 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:53.861731 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 392/794] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:23:53.865365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a213896-d219-4f2f-a2d4-48049be39235 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:53.895797 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c01c8-6d32-4b08-ad48-fef326e710f3 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:53.988484 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec127e-0e5d-4a32-b7d7-27d0aa723543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:53.989136 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec127e-0e5d-4a32-b7d7-27d0aa723543 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:53.989423 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec127e-0e5d-4a32-b7d7-27d0aa723543 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:53.989704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec127e-0e5d-4a32-b7d7-27d0aa723543 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:54.011757 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a213896-d219-4f2f-a2d4-48049be39235 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:54.016901 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec127e-0e5d-4a32-b7d7-27d0aa723543 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:54.021629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-c2ec127e-0e5d-4a32-b7d7-27d0aa723543 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:23:54.021629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:54.021629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:54.021629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:54.021629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:54.021629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:54.021629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:54.021629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:54.021629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:54.021629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:54.021629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:54.021629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:54.021629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:54.021629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:54.021629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:54.021629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:54.023122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:54.023122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:54.023122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:54.023122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:54.023122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:54.023122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:54.023122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:54.023122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:54.023122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:54.023122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:54.023122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:54.023122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:54.023122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:54.023122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:54.023122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:54.023122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:54.024038 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:54.024038 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:54.024038 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:54.024038 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:54.024038 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:54.024038 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:54.024038 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:54.024038 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:54.024038 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:54.024038 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:54.024038 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:23:54.024038 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:54.032141 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 403/795] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:23:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:23:54.053332 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:54.054862 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec127e-0e5d-4a32-b7d7-27d0aa723543 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:54.060634 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0b2ef-4f5a-4711-98e5-e62bc537fbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:54.061472 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0b2ef-4f5a-4711-98e5-e62bc537fbe0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:54.061997 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0b2ef-4f5a-4711-98e5-e62bc537fbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:54.062453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0b2ef-4f5a-4711-98e5-e62bc537fbe0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:54.070836 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0b2ef-4f5a-4711-98e5-e62bc537fbe0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:54.072631 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0b2ef-4f5a-4711-98e5-e62bc537fbe0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:54.085498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-04a0b2ef-4f5a-4711-98e5-e62bc537fbe0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 12:23:54.085498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:54.085498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:54.085498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:54.085498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:54.085498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:54.085498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:54.085498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:54.085498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:54.085498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:54.085498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:54.085498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:54.085498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:23:54.085498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:23:54.085498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:23:54.085498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:23:54.086463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:54.086463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:54.086463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:23:54.086463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:54.086463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:23:54.086463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:54.086463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:54.086463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:54.086463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:54.086463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:54.086463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:54.086463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:54.086463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:54.086463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:54.086463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:54.086463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:54.086463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:54.086463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:54.087427 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:54.087427 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:54.087427 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:23:54.087427 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:23:54.087427 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:23:54.087427 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:23:54.087427 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:23:54.087427 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:23:54.087427 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 12:23:54.087427 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:54.089548 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 393/796] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 10 12:23:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:54.100796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0b2ef-4f5a-4711-98e5-e62bc537fbe0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:54.117724 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:54.220011 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0b2ef-4f5a-4711-98e5-e62bc537fbe0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:54.246326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec127e-0e5d-4a32-b7d7-27d0aa723543 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:54.264206 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d517e72-94b6-4002-9ce5-c03b9cdbe069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:54.265257 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d517e72-94b6-4002-9ce5-c03b9cdbe069 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:54.265687 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d517e72-94b6-4002-9ce5-c03b9cdbe069 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:54.266112 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d517e72-94b6-4002-9ce5-c03b9cdbe069 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:54.267109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d517e72-94b6-4002-9ce5-c03b9cdbe069 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:54.267582 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d517e72-94b6-4002-9ce5-c03b9cdbe069 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:54.269218 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d517e72-94b6-4002-9ce5-c03b9cdbe069 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:54.295498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 394/797] 10.222.0.49 () {56 vars in 1149 bytes} [Wed Aug 10 12:23:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:54.304024 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d517e72-94b6-4002-9ce5-c03b9cdbe069 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:54.327941 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dabfbce-0be1-4c39-a16f-037ae79aac3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:54.328904 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabfbce-0be1-4c39-a16f-037ae79aac3e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:54.328904 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabfbce-0be1-4c39-a16f-037ae79aac3e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:54.329096 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabfbce-0be1-4c39-a16f-037ae79aac3e None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:54.329760 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dabfbce-0be1-4c39-a16f-037ae79aac3e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:54.330067 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dabfbce-0be1-4c39-a16f-037ae79aac3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:54.331462 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dabfbce-0be1-4c39-a16f-037ae79aac3e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:54.351115 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 404/798] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:23:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:54.363749 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:54.369538 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dabfbce-0be1-4c39-a16f-037ae79aac3e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:54.421497 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d517e72-94b6-4002-9ce5-c03b9cdbe069 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:54.459098 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b9c5c-a185-42df-9572-4ac3b7a84e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:54.460597 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b9c5c-a185-42df-9572-4ac3b7a84e24 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:54.461124 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b9c5c-a185-42df-9572-4ac3b7a84e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:54.461553 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b9c5c-a185-42df-9572-4ac3b7a84e24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:54.473321 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b9c5c-a185-42df-9572-4ac3b7a84e24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:54.475264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b9c5c-a185-42df-9572-4ac3b7a84e24 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:54.491144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-682b9c5c-a185-42df-9572-4ac3b7a84e24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:54.491144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:54.491144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:54.491144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:54.491144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:54.491144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:54.491144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:54.491144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:54.491144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:54.491144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:54.491144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:54.491144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:54.491144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:54.491144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:54.491144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:54.491144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:54.492152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:54.492152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:54.492152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:54.492152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:54.492152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:54.492152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:54.492152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:54.492152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:54.492152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:54.492152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:54.492152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:54.492152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:54.492152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:54.492152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:54.492152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:54.492152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:54.492152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:54.493061 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:54.493061 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:54.493061 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:54.495282 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 395/799] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:54.500724 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b9c5c-a185-42df-9572-4ac3b7a84e24 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:54.578685 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dabfbce-0be1-4c39-a16f-037ae79aac3e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:54.587531 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:54.653824 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ab5f6-0270-4b03-8cf5-9e2542316371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:54.654386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ab5f6-0270-4b03-8cf5-9e2542316371 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:54.654666 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ab5f6-0270-4b03-8cf5-9e2542316371 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:54.654939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ab5f6-0270-4b03-8cf5-9e2542316371 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:54.675699 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ab5f6-0270-4b03-8cf5-9e2542316371 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:54.677161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ab5f6-0270-4b03-8cf5-9e2542316371 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:54.700323 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b9c5c-a185-42df-9572-4ac3b7a84e24 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:54.702496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:54.708822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-650ab5f6-0270-4b03-8cf5-9e2542316371 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:54.708822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:54.708822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:54.708822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:54.708822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:54.708822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:54.708822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:54.708822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:54.708822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:54.708822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:54.708822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:54.708822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:54.708822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:54.708822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:54.708822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:54.708822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:54.710048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:54.710048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:54.710048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:54.710048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:54.710048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:54.710048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:54.710048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:54.710048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:54.710048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:54.710048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:54.710048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:54.710048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:54.710048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:54.710048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:54.710048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:54.710048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:54.710048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:54.710967 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:54.710967 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:54.710967 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:54.710967 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 405/800] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:54.711820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6a1c7-b6dd-4eaf-9a9b-ac6a251c3b51 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:54.712099 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6a1c7-b6dd-4eaf-9a9b-ac6a251c3b51 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:54.717285 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6a1c7-b6dd-4eaf-9a9b-ac6a251c3b51 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:54.718317 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 406/801] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:54.722336 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6a1c7-b6dd-4eaf-9a9b-ac6a251c3b51 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:54.747976 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84ac7c-f98e-4037-8838-24e53820ef3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:54.748609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84ac7c-f98e-4037-8838-24e53820ef3e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:54.748886 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84ac7c-f98e-4037-8838-24e53820ef3e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:54.749156 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84ac7c-f98e-4037-8838-24e53820ef3e None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:54.749895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84ac7c-f98e-4037-8838-24e53820ef3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:54.750202 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84ac7c-f98e-4037-8838-24e53820ef3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:54.751608 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84ac7c-f98e-4037-8838-24e53820ef3e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:54.765022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 396/802] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:54.766477 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238a717-7353-4507-98a0-1e403f172bfa None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:54.766868 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238a717-7353-4507-98a0-1e403f172bfa None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:54.767143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238a717-7353-4507-98a0-1e403f172bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:54.792232 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0238a717-7353-4507-98a0-1e403f172bfa None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:54.793561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0238a717-7353-4507-98a0-1e403f172bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:54.845963 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6a1c7-b6dd-4eaf-9a9b-ac6a251c3b51 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:54.876453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 397/803] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:23:54.883429 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238a717-7353-4507-98a0-1e403f172bfa None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:54.892963 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b3e91-0f13-45ec-9220-88b1e8b3cc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:54.894113 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b3e91-0f13-45ec-9220-88b1e8b3cc54 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:54.894542 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b3e91-0f13-45ec-9220-88b1e8b3cc54 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:54.894993 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b3e91-0f13-45ec-9220-88b1e8b3cc54 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:54.895925 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b3e91-0f13-45ec-9220-88b1e8b3cc54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:54.898041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b3e91-0f13-45ec-9220-88b1e8b3cc54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:54.900430 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b3e91-0f13-45ec-9220-88b1e8b3cc54 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:54.913249 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 407/804] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:54.914381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141d627-0d06-4816-9465-0076f32dfa35 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:54.914670 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141d627-0d06-4816-9465-0076f32dfa35 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:54.915032 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141d627-0d06-4816-9465-0076f32dfa35 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:54.916415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 408/805] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:54.916953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141d627-0d06-4816-9465-0076f32dfa35 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:54.933409 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:54.982447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:54.991681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238a717-7353-4507-98a0-1e403f172bfa None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:55.028236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fbc48-b9e6-4bb6-80ba-f61d71490874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:55.028906 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fbc48-b9e6-4bb6-80ba-f61d71490874 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:55.029131 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fbc48-b9e6-4bb6-80ba-f61d71490874 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:55.029404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fbc48-b9e6-4bb6-80ba-f61d71490874 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:55.033127 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141d627-0d06-4816-9465-0076f32dfa35 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:55.036454 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195fbc48-b9e6-4bb6-80ba-f61d71490874 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:55.038031 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195fbc48-b9e6-4bb6-80ba-f61d71490874 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:55.050715 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-195fbc48-b9e6-4bb6-80ba-f61d71490874 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:55.050715 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:55.050715 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:55.050715 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:55.050715 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:55.050715 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:55.050715 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:55.050715 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:55.050715 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:55.050715 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:55.050715 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:55.050715 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:55.050715 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:23:55.050715 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:23:55.050715 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:23:55.054507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:23:55.054507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:55.054507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:55.054507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:55.054507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:55.054507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:55.054507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:55.054507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:55.054507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:55.054507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:55.054507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:55.054507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:55.054507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:55.054507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:55.054507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:55.054507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:55.054507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:55.055437 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:23:55.055437 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:23:55.055437 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:23:55.055437 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:23:55.055437 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:23:55.055437 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:23:55.055437 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:55.055437 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:55.055437 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 398/806] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:55.055437 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fbc48-b9e6-4bb6-80ba-f61d71490874 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:55.073832 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa28f4-51c0-4c3a-9a25-fc3945d55c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:55.074469 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa28f4-51c0-4c3a-9a25-fc3945d55c08 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:55.074750 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa28f4-51c0-4c3a-9a25-fc3945d55c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:55.075349 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa28f4-51c0-4c3a-9a25-fc3945d55c08 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:55.086290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aa28f4-51c0-4c3a-9a25-fc3945d55c08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:55.090958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-15aa28f4-51c0-4c3a-9a25-fc3945d55c08 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:23:55.090958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:55.090958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:55.090958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:55.090958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:55.090958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:55.090958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:55.090958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:55.090958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:55.090958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:55.090958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:55.090958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:55.090958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:55.090958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:55.090958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:55.090958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:55.092027 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:55.092027 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:55.092027 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:55.092027 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:55.092027 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:55.092027 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:55.092027 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:55.092027 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:55.092027 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:55.092027 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:55.092027 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:55.092027 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:55.092027 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:55.092027 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:55.092027 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:55.092027 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:55.093365 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:55.093365 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:55.093365 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:55.093365 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:55.093365 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:55.093365 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:55.093365 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:55.093365 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:55.093365 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:55.093365 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:55.093365 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:23:55.093365 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:55.093365 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 409/807] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:23:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:55.094073 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222bc84-7d3b-415f-a64f-9e4ea457fc82 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:55.094073 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222bc84-7d3b-415f-a64f-9e4ea457fc82 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:55.094073 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222bc84-7d3b-415f-a64f-9e4ea457fc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:55.112154 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7222bc84-7d3b-415f-a64f-9e4ea457fc82 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:55.113361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7222bc84-7d3b-415f-a64f-9e4ea457fc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:55.178695 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fbc48-b9e6-4bb6-80ba-f61d71490874 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:55.180522 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:55.185239 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 410/808] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:23:55.186046 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7222bc84-7d3b-415f-a64f-9e4ea457fc82 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:55.228816 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846eb67b-e053-46b2-9c96-efb87efea0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:55.229454 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846eb67b-e053-46b2-9c96-efb87efea0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:55.229737 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846eb67b-e053-46b2-9c96-efb87efea0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:55.230358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846eb67b-e053-46b2-9c96-efb87efea0f7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:55.237247 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846eb67b-e053-46b2-9c96-efb87efea0f7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:55.238712 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846eb67b-e053-46b2-9c96-efb87efea0f7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:55.250761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-846eb67b-e053-46b2-9c96-efb87efea0f7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 12:23:55.250761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:55.250761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:55.250761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:55.250761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:55.250761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:55.250761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:55.250761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:55.250761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:55.250761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:55.250761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:55.250761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:55.250761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:23:55.250761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:23:55.250761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:23:55.250761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:23:55.251914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:55.251914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:55.251914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:23:55.251914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:55.251914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:23:55.251914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:55.251914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:55.251914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:55.251914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:55.251914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:55.251914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:55.251914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:55.251914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:55.251914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:55.251914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:55.251914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:55.251914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:55.251914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:55.252868 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:55.252868 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:55.252868 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:23:55.252868 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:23:55.252868 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:23:55.252868 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:23:55.252868 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:23:55.252868 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:23:55.252868 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 12:23:55.252868 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:55.252868 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 399/809] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:23:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:55.252868 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846eb67b-e053-46b2-9c96-efb87efea0f7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:55.270151 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:55.300989 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7222bc84-7d3b-415f-a64f-9e4ea457fc82 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:55.341174 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d9242-0e7e-48b8-bb46-79bc3dba21f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:55.342150 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d9242-0e7e-48b8-bb46-79bc3dba21f7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:55.342150 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d9242-0e7e-48b8-bb46-79bc3dba21f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:55.342425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d9242-0e7e-48b8-bb46-79bc3dba21f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:55.343099 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d9242-0e7e-48b8-bb46-79bc3dba21f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:55.343438 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d9242-0e7e-48b8-bb46-79bc3dba21f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:55.344869 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d9242-0e7e-48b8-bb46-79bc3dba21f7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:55.362251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 411/810] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:23:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:55.362251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d9242-0e7e-48b8-bb46-79bc3dba21f7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:55.374165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846eb67b-e053-46b2-9c96-efb87efea0f7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:55.409713 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ad3b7-ac05-40dd-a1af-bc659aac97cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:55.410330 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ad3b7-ac05-40dd-a1af-bc659aac97cb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:55.410613 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ad3b7-ac05-40dd-a1af-bc659aac97cb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:55.410886 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ad3b7-ac05-40dd-a1af-bc659aac97cb None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:55.411590 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1ad3b7-ac05-40dd-a1af-bc659aac97cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:55.411895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1ad3b7-ac05-40dd-a1af-bc659aac97cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:55.413536 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1ad3b7-ac05-40dd-a1af-bc659aac97cb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:55.426419 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 400/811] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:23:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:55.427315 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ad3b7-ac05-40dd-a1af-bc659aac97cb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:55.435162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:55.487490 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d9242-0e7e-48b8-bb46-79bc3dba21f7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:55.505452 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:55.524856 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193fc2d-dc8c-4892-b110-5e551de108df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:55.525555 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193fc2d-dc8c-4892-b110-5e551de108df None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:55.525840 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193fc2d-dc8c-4892-b110-5e551de108df None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:55.526117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193fc2d-dc8c-4892-b110-5e551de108df None admin] PATH_INFO: `/v3/services/network` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:55.526726 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193fc2d-dc8c-4892-b110-5e551de108df None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:55.528181 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193fc2d-dc8c-4892-b110-5e551de108df None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:55.535777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ad3b7-ac05-40dd-a1af-bc659aac97cb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:55.537095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-c193fc2d-dc8c-4892-b110-5e551de108df None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 12:23:55.537095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:55.537095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:23:55.537095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:23:55.537095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:23:55.537095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:23:55.537095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:23:55.537095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:23:55.537095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 12:23:55.537095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:55.537095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:23:55.537095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:55.537095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:55.537095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:55.537095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:55.537095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:55.537095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:55.538886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:55.538886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:55.538886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:55.538886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:55.538886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:55.538886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:55.538886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:23:55.538886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:23:55.538886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:23:55.538886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:23:55.538886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:55.538886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:55.538886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:55.538886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:55.538886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:55.538886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:55.539839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:55.539839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:55.539839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:55.539839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:55.539839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:55.539839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:55.539839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:55.539839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:55.539839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:55.539839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:55.539839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:23:55.539839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:23:55.539839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 12:23:55.539839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:55.539839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 412/812] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:55 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:55.540832 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193fc2d-dc8c-4892-b110-5e551de108df None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:55.575385 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ee6a8-06cd-4769-97d9-99db163b79f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:55.575993 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ee6a8-06cd-4769-97d9-99db163b79f4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:55.576271 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ee6a8-06cd-4769-97d9-99db163b79f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:55.576541 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ee6a8-06cd-4769-97d9-99db163b79f4 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:55.577250 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ee6a8-06cd-4769-97d9-99db163b79f4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:55.577550 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ee6a8-06cd-4769-97d9-99db163b79f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:55.578969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ee6a8-06cd-4769-97d9-99db163b79f4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:55.595671 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 401/813] 10.222.0.49 () {56 vars in 1135 bytes} [Wed Aug 10 12:23:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:55.596441 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ee6a8-06cd-4769-97d9-99db163b79f4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:55.641984 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193fc2d-dc8c-4892-b110-5e551de108df None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:55.664717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:55.675877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de32de9-f6fe-4f66-ac4f-a1ef7b1a7c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:55.676536 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de32de9-f6fe-4f66-ac4f-a1ef7b1a7c46 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:55.676813 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de32de9-f6fe-4f66-ac4f-a1ef7b1a7c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:55.677091 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de32de9-f6fe-4f66-ac4f-a1ef7b1a7c46 None admin] PATH_INFO: `/v3/projects/8e201e5f281a4977be503e22daf149d7/users/e42059e9af9842548ea08dcd39ff0b6e/roles/a9bbb011afca4176a3def9356cb5f165` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:55.692446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de32de9-f6fe-4f66-ac4f-a1ef7b1a7c46 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e201e5f281a4977be503e22daf149d7, user_id=e42059e9af9842548ea08dcd39ff0b6e, role_id=a9bbb011afca4176a3def9356cb5f165)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:55.694444 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de32de9-f6fe-4f66-ac4f-a1ef7b1a7c46 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:55.696408 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ee6a8-06cd-4769-97d9-99db163b79f4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:55.723877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:55.729949 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a018222-04a7-49ce-b3cb-38fcc3a61471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:55.730573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a018222-04a7-49ce-b3cb-38fcc3a61471 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:55.730851 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a018222-04a7-49ce-b3cb-38fcc3a61471 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:55.731128 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a018222-04a7-49ce-b3cb-38fcc3a61471 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:55.741430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a018222-04a7-49ce-b3cb-38fcc3a61471 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:55.742849 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a018222-04a7-49ce-b3cb-38fcc3a61471 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:55.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-1a018222-04a7-49ce-b3cb-38fcc3a61471 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:55.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:55.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:55.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:55.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:55.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:55.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:55.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:55.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:55.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:55.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:55.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:55.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:55.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:55.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:55.757789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:55.760759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:55.760759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:55.760759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:55.760759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:55.760759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:55.760759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:55.760759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:55.760759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:55.760759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:55.760759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:55.760759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:55.760759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:55.760759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:55.760759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:55.760759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:55.760759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:55.760759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:55.761678 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:55.761678 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:55.761678 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:55.761678 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 402/814] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:55.761678 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9675f7-ef33-467b-bc00-ffb0eb1abbbb None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:55.761678 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9675f7-ef33-467b-bc00-ffb0eb1abbbb None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:55.761678 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9675f7-ef33-467b-bc00-ffb0eb1abbbb None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:55.763414 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 413/815] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 12:23:55 2022] PUT /identity/v3/projects/8e201e5f281a4977be503e22daf149d7/users/e42059e9af9842548ea08dcd39ff0b6e/roles/a9bbb011afca4176a3def9356cb5f165 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:23:55.763414 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de32de9-f6fe-4f66-ac4f-a1ef7b1a7c46 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:55.763573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 403/816] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:55.763573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9675f7-ef33-467b-bc00-ffb0eb1abbbb None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:55.887140 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de32de9-f6fe-4f66-ac4f-a1ef7b1a7c46 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:55.888690 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9675f7-ef33-467b-bc00-ffb0eb1abbbb None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:55.929450 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e16c6a-05cd-4126-aed7-63b02e9c3d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:55.930057 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f73c84-b2a1-4f77-9392-7f37f5896bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:55.930507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e16c6a-05cd-4126-aed7-63b02e9c3d34 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:55.930507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e16c6a-05cd-4126-aed7-63b02e9c3d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:55.930641 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e16c6a-05cd-4126-aed7-63b02e9c3d34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:55.930709 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f73c84-b2a1-4f77-9392-7f37f5896bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:55.930953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f73c84-b2a1-4f77-9392-7f37f5896bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:55.931256 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f73c84-b2a1-4f77-9392-7f37f5896bb7 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:55.931887 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f73c84-b2a1-4f77-9392-7f37f5896bb7 None admin] RBAC: Adding query filter params (name=network) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:55.932192 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f73c84-b2a1-4f77-9392-7f37f5896bb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:55.933543 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f73c84-b2a1-4f77-9392-7f37f5896bb7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:55.939101 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 414/817] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 10 12:23:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:55.940572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f5aa3-c159-410d-8218-91255dd28c41 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:55.941111 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f5aa3-c159-410d-8218-91255dd28c41 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:55.941590 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f5aa3-c159-410d-8218-91255dd28c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:55.941876 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e16c6a-05cd-4126-aed7-63b02e9c3d34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:55.943456 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e16c6a-05cd-4126-aed7-63b02e9c3d34 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:55.958507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-35e16c6a-05cd-4126-aed7-63b02e9c3d34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:55.958507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:55.958507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:55.958507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:55.958507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:55.958507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:55.958507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:55.958507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:55.958507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:55.958507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:55.958507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:55.958507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:55.958507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:55.958507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:55.958507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:55.958507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:55.962121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:55.962121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:55.962121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:55.962121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:55.962121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:55.962121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:55.962121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:55.962121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:55.962121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:55.962121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:55.962121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:55.962121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:55.962121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:55.962121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:55.962121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:55.962121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:55.962121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:55.963022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:55.963022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:55.963022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:55.963022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 404/818] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:55.963022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e16c6a-05cd-4126-aed7-63b02e9c3d34 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:55.963352 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-261f5aa3-c159-410d-8218-91255dd28c41 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:55.963352 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-261f5aa3-c159-410d-8218-91255dd28c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:55.997809 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:56.001671 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:56.042697 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 415/819] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:23:56.043742 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f5aa3-c159-410d-8218-91255dd28c41 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:56.078184 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e16c6a-05cd-4126-aed7-63b02e9c3d34 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:56.115610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902df3b-b494-49f2-b08b-58a84d1f3d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:56.116771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902df3b-b494-49f2-b08b-58a84d1f3d53 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:56.117144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902df3b-b494-49f2-b08b-58a84d1f3d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:56.117418 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902df3b-b494-49f2-b08b-58a84d1f3d53 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:56.118051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3902df3b-b494-49f2-b08b-58a84d1f3d53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:56.118358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3902df3b-b494-49f2-b08b-58a84d1f3d53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:56.119741 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3902df3b-b494-49f2-b08b-58a84d1f3d53 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:56.136389 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 405/820] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:56.136389 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902df3b-b494-49f2-b08b-58a84d1f3d53 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:56.160186 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f5aa3-c159-410d-8218-91255dd28c41 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:56.196987 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eea4c9-10bb-4ab6-a40f-307c69d3c1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:56.197520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eea4c9-10bb-4ab6-a40f-307c69d3c1cb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:56.197826 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eea4c9-10bb-4ab6-a40f-307c69d3c1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:56.198095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eea4c9-10bb-4ab6-a40f-307c69d3c1cb None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:56.198719 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eea4c9-10bb-4ab6-a40f-307c69d3c1cb None admin] RBAC: Adding query filter params (type=network) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:56.199024 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eea4c9-10bb-4ab6-a40f-307c69d3c1cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:56.200403 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eea4c9-10bb-4ab6-a40f-307c69d3c1cb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:56.205755 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 416/821] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 10 12:23:56 2022] GET /identity/v3/services?type=network => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:56.206417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eea4c9-10bb-4ab6-a40f-307c69d3c1cb None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:56.239060 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:56.239354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:56.246875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902df3b-b494-49f2-b08b-58a84d1f3d53 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:56.287206 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddaf1a-c37a-466e-9699-a7f86bb2b985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:56.287886 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddaf1a-c37a-466e-9699-a7f86bb2b985 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:56.288272 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddaf1a-c37a-466e-9699-a7f86bb2b985 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:56.288549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddaf1a-c37a-466e-9699-a7f86bb2b985 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:56.289193 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ddaf1a-c37a-466e-9699-a7f86bb2b985 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:56.289499 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ddaf1a-c37a-466e-9699-a7f86bb2b985 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:56.290871 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ddaf1a-c37a-466e-9699-a7f86bb2b985 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:56.307555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 406/822] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:56.307555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddaf1a-c37a-466e-9699-a7f86bb2b985 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:56.320855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eea4c9-10bb-4ab6-a40f-307c69d3c1cb None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:56.356860 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da128e-9d23-4a9d-9f4c-ed6b1119eb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:56.357485 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da128e-9d23-4a9d-9f4c-ed6b1119eb10 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:56.357762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da128e-9d23-4a9d-9f4c-ed6b1119eb10 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:56.358033 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da128e-9d23-4a9d-9f4c-ed6b1119eb10 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:56.358611 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75da128e-9d23-4a9d-9f4c-ed6b1119eb10 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:56.359994 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75da128e-9d23-4a9d-9f4c-ed6b1119eb10 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:56.368944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-75da128e-9d23-4a9d-9f4c-ed6b1119eb10 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 12:23:56.368944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:56.368944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:23:56.368944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:23:56.368944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:23:56.368944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:23:56.368944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:23:56.368944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:23:56.368944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 12:23:56.368944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:56.368944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:23:56.368944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:56.368944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:56.368944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:56.368944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:56.368944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:56.368944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:56.369982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:56.369982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:56.369982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:56.369982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:56.369982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:56.369982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:56.369982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:23:56.369982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:23:56.369982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:23:56.369982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:23:56.369982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:56.369982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:56.369982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:56.369982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:56.369982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:56.369982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:56.370839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:56.370839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:56.370839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:56.370839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:56.370839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:56.370839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:56.370839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:56.370839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:56.370839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:56.370839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:56.370839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:23:56.370839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:23:56.370839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 12:23:56.370839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:56.370839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 417/823] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 12:23:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:56.371965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da128e-9d23-4a9d-9f4c-ed6b1119eb10 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:56.427974 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddaf1a-c37a-466e-9699-a7f86bb2b985 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:56.463750 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f402e85-b4d9-4741-b202-c11d76a857a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:56.464304 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f402e85-b4d9-4741-b202-c11d76a857a2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:56.464580 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f402e85-b4d9-4741-b202-c11d76a857a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:56.464853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f402e85-b4d9-4741-b202-c11d76a857a2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:56.472686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f402e85-b4d9-4741-b202-c11d76a857a2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:56.474115 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f402e85-b4d9-4741-b202-c11d76a857a2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:56.475102 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:56.481526 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da128e-9d23-4a9d-9f4c-ed6b1119eb10 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:56.486421 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-9f402e85-b4d9-4741-b202-c11d76a857a2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:23:56.486421 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:56.486421 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:56.486421 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:56.486421 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:56.486421 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:56.486421 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:56.486421 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:56.486421 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:56.486421 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:56.486421 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:56.486421 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:56.486421 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:23:56.486421 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:23:56.486421 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:23:56.486421 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:23:56.488016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:56.488016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:56.488016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:23:56.488016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:56.488016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:23:56.488016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:56.488016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:56.488016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:56.488016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:56.488016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:56.488016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:56.488016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:56.488016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:56.488016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:56.488016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:56.488016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:56.488016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:56.488016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:56.489439 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:56.489439 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:56.489439 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:23:56.489439 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:23:56.489439 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:23:56.489439 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:23:56.489439 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:23:56.489439 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:23:56.489439 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:23:56.489439 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:56.492707 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 407/824] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Aug 10 12:23:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:56.493704 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f402e85-b4d9-4741-b202-c11d76a857a2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:56.511854 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:56.519152 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb7c04-378b-4872-b990-d7a0545f3c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:56.519856 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb7c04-378b-4872-b990-d7a0545f3c2d None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:56.520194 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb7c04-378b-4872-b990-d7a0545f3c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:56.520472 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb7c04-378b-4872-b990-d7a0545f3c2d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:56.521172 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeb7c04-378b-4872-b990-d7a0545f3c2d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:56.523158 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeb7c04-378b-4872-b990-d7a0545f3c2d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:56.598130 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f402e85-b4d9-4741-b202-c11d76a857a2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:56.602948 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 418/825] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:23:56 2022] POST /identity/v3/endpoints => generated 337 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:23:56.603990 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb7c04-378b-4872-b990-d7a0545f3c2d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:56.671526 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd79869-879e-45ba-b7f1-59e12814ee92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:56.672700 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd79869-879e-45ba-b7f1-59e12814ee92 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:56.673190 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd79869-879e-45ba-b7f1-59e12814ee92 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:56.673857 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd79869-879e-45ba-b7f1-59e12814ee92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:56.686081 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd79869-879e-45ba-b7f1-59e12814ee92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:56.690006 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd79869-879e-45ba-b7f1-59e12814ee92 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:56.708908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-5dd79869-879e-45ba-b7f1-59e12814ee92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:56.708908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:56.708908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:56.708908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:56.708908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:56.708908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:56.708908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:56.708908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:56.708908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:56.708908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:56.708908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:56.708908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:56.708908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:23:56.708908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:23:56.708908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:23:56.710088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:23:56.710088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:56.710088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:56.710088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:56.710088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:56.710088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:56.710088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:56.710088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:56.710088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:56.710088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:56.710088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:56.710088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:56.710088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:56.710088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:56.710088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:56.710088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:56.710088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:56.711290 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:23:56.711290 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:23:56.711290 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:23:56.711290 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:23:56.711290 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:23:56.711290 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:23:56.711290 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:56.711290 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:56.714340 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 408/826] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:56.715404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd79869-879e-45ba-b7f1-59e12814ee92 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:56.733525 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb7c04-378b-4872-b990-d7a0545f3c2d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:56.778825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b91c20-7b6f-4cbc-a0de-38d7b0a5b261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:56.779507 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b91c20-7b6f-4cbc-a0de-38d7b0a5b261 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:56.779787 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b91c20-7b6f-4cbc-a0de-38d7b0a5b261 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:56.780058 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b91c20-7b6f-4cbc-a0de-38d7b0a5b261 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:56.780710 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b91c20-7b6f-4cbc-a0de-38d7b0a5b261 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:56.781016 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b91c20-7b6f-4cbc-a0de-38d7b0a5b261 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:56.782375 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b91c20-7b6f-4cbc-a0de-38d7b0a5b261 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:56.789795 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 419/827] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 12:23:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:56.791143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b91c20-7b6f-4cbc-a0de-38d7b0a5b261 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:56.811337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:56.825098 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:56.859716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd79869-879e-45ba-b7f1-59e12814ee92 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:56.898239 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8bd0c0-4a72-4570-a50a-a1657fb24d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:56.898842 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8bd0c0-4a72-4570-a50a-a1657fb24d25 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:56.899126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8bd0c0-4a72-4570-a50a-a1657fb24d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:56.899424 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8bd0c0-4a72-4570-a50a-a1657fb24d25 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:56.900269 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8bd0c0-4a72-4570-a50a-a1657fb24d25 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:56.900599 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8bd0c0-4a72-4570-a50a-a1657fb24d25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:56.902019 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8bd0c0-4a72-4570-a50a-a1657fb24d25 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:56.924153 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 409/828] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Aug 10 12:23:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:56.924153 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8bd0c0-4a72-4570-a50a-a1657fb24d25 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:56.930660 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b91c20-7b6f-4cbc-a0de-38d7b0a5b261 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:56.969562 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e2ef3-7706-40c6-bbcc-2f52846e7c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:56.970068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e2ef3-7706-40c6-bbcc-2f52846e7c48 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:56.970346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e2ef3-7706-40c6-bbcc-2f52846e7c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:56.970621 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e2ef3-7706-40c6-bbcc-2f52846e7c48 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:56.971757 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732e2ef3-7706-40c6-bbcc-2f52846e7c48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:56.972067 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732e2ef3-7706-40c6-bbcc-2f52846e7c48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:56.973850 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732e2ef3-7706-40c6-bbcc-2f52846e7c48 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:56.994879 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 420/829] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:23:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:56.994879 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e2ef3-7706-40c6-bbcc-2f52846e7c48 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:57.034571 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8bd0c0-4a72-4570-a50a-a1657fb24d25 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:57.064764 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:57.073163 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebbc554-13d5-462d-b3dd-da7c2e566e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:57.073772 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbc554-13d5-462d-b3dd-da7c2e566e57 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:57.074051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbc554-13d5-462d-b3dd-da7c2e566e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:57.074326 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbc554-13d5-462d-b3dd-da7c2e566e57 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:57.074946 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebbc554-13d5-462d-b3dd-da7c2e566e57 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:57.075278 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebbc554-13d5-462d-b3dd-da7c2e566e57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:57.076645 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebbc554-13d5-462d-b3dd-da7c2e566e57 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:57.078058 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:57.081545 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 410/830] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 12:23:56 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:57.082784 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebbc554-13d5-462d-b3dd-da7c2e566e57 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:57.124646 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e2ef3-7706-40c6-bbcc-2f52846e7c48 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:57.176412 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caff891-9d82-413c-b99d-1e49c93e3a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:57.177682 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caff891-9d82-413c-b99d-1e49c93e3a4b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:57.178730 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caff891-9d82-413c-b99d-1e49c93e3a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:57.179495 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caff891-9d82-413c-b99d-1e49c93e3a4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:57.198875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebbc554-13d5-462d-b3dd-da7c2e566e57 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:57.247847 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958dc32-0a5e-46f9-962f-73e5ec3c2e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:57.248927 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958dc32-0a5e-46f9-962f-73e5ec3c2e05 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:57.249286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958dc32-0a5e-46f9-962f-73e5ec3c2e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:57.249567 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958dc32-0a5e-46f9-962f-73e5ec3c2e05 None admin] PATH_INFO: `/v3/projects/8e201e5f281a4977be503e22daf149d7/users/59395bfa505c4540b3957deee78eb698/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:57.252855 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958dc32-0a5e-46f9-962f-73e5ec3c2e05 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e201e5f281a4977be503e22daf149d7, user_id=59395bfa505c4540b3957deee78eb698, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:57.254292 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958dc32-0a5e-46f9-962f-73e5ec3c2e05 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:57.273412 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caff891-9d82-413c-b99d-1e49c93e3a4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:57.274858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caff891-9d82-413c-b99d-1e49c93e3a4b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:57.344796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 411/831] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 12:23:57 2022] PUT /identity/v3/projects/8e201e5f281a4977be503e22daf149d7/users/59395bfa505c4540b3957deee78eb698/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:23:57.360508 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:57.393546 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 421/832] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:23:56 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:23:57.395842 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958dc32-0a5e-46f9-962f-73e5ec3c2e05 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:57.473729 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:57.513461 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958dc32-0a5e-46f9-962f-73e5ec3c2e05 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:57.549618 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7be0fd-dd49-4740-b20a-9508b1fedb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:57.550260 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7be0fd-dd49-4740-b20a-9508b1fedb00 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:57.550559 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7be0fd-dd49-4740-b20a-9508b1fedb00 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:57.550823 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7be0fd-dd49-4740-b20a-9508b1fedb00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:57.562171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7be0fd-dd49-4740-b20a-9508b1fedb00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:57.563667 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7be0fd-dd49-4740-b20a-9508b1fedb00 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:57.580110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-4b7be0fd-dd49-4740-b20a-9508b1fedb00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:57.580110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:57.580110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:57.580110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:57.580110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:57.580110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:57.580110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:57.580110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:57.580110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:57.580110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:57.580110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:57.580110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:57.580110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:57.580110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:57.580110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:57.580110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:57.581097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:57.581097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:57.581097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:57.581097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:57.581097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:57.581097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:57.581097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:57.581097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:57.581097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:57.581097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:57.581097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:57.581097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:57.581097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:57.581097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:57.581097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:57.581097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:57.581097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:57.581986 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:57.581986 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:57.581986 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:57.585119 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 412/833] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:57.587989 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caff891-9d82-413c-b99d-1e49c93e3a4b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:57.692913 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caff891-9d82-413c-b99d-1e49c93e3a4b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:57.728697 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193299b-8418-4408-bec8-6e0f60c0b4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:57.729309 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193299b-8418-4408-bec8-6e0f60c0b4f9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:57.729587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193299b-8418-4408-bec8-6e0f60c0b4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:57.729860 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193299b-8418-4408-bec8-6e0f60c0b4f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:57.730498 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193299b-8418-4408-bec8-6e0f60c0b4f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:57.730802 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193299b-8418-4408-bec8-6e0f60c0b4f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:57.732193 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193299b-8418-4408-bec8-6e0f60c0b4f9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:57.748844 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 422/834] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:57.750291 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7be0fd-dd49-4740-b20a-9508b1fedb00 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:57.815882 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc7932-2661-4258-97d3-d8c4ba320ddb None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:57.816167 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc7932-2661-4258-97d3-d8c4ba320ddb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:57.816365 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc7932-2661-4258-97d3-d8c4ba320ddb None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:57.818071 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 423/835] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:23:57.823954 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f56b1-c8b4-45ea-9eee-9adc5801c17c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:57.824368 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f56b1-c8b4-45ea-9eee-9adc5801c17c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:57.824740 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f56b1-c8b4-45ea-9eee-9adc5801c17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:57.849218 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-631f56b1-c8b4-45ea-9eee-9adc5801c17c None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:57.851437 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-631f56b1-c8b4-45ea-9eee-9adc5801c17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:57.863268 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7be0fd-dd49-4740-b20a-9508b1fedb00 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:57.866623 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:57.896893 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:57.909715 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64adc41e-a0f4-4e8c-af45-7b8090450b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:57.911041 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64adc41e-a0f4-4e8c-af45-7b8090450b6a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:57.911949 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64adc41e-a0f4-4e8c-af45-7b8090450b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:57.912628 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64adc41e-a0f4-4e8c-af45-7b8090450b6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:57.921329 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64adc41e-a0f4-4e8c-af45-7b8090450b6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:57.923740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64adc41e-a0f4-4e8c-af45-7b8090450b6a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:57.939815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-64adc41e-a0f4-4e8c-af45-7b8090450b6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:57.939815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:57.939815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:57.939815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:57.939815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:57.939815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:57.939815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:57.939815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:57.939815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:57.939815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:57.939815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:57.939815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:57.939815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:23:57.939815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:23:57.939815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:23:57.940736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:23:57.940736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:57.940736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:57.940736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:57.940736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:57.940736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:57.940736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:57.940736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:57.940736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:57.940736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:57.940736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:57.940736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:57.940736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:57.940736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:57.940736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:57.940736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:57.940736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:57.941611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:23:57.941611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:23:57.941611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:23:57.941611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:23:57.941611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:23:57.941611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:23:57.941611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:57.941611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:57.943528 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 424/836] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:23:57.949609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 413/837] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:57.951124 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f56b1-c8b4-45ea-9eee-9adc5801c17c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:57.964446 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64adc41e-a0f4-4e8c-af45-7b8090450b6a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:58.064089 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f56b1-c8b4-45ea-9eee-9adc5801c17c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:58.070625 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64adc41e-a0f4-4e8c-af45-7b8090450b6a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:58.102087 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf0adc-79f4-425a-826b-78cd42616905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:58.102485 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf0adc-79f4-425a-826b-78cd42616905 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:58.102771 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf0adc-79f4-425a-826b-78cd42616905 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:58.103041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf0adc-79f4-425a-826b-78cd42616905 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:58.103823 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf0adc-79f4-425a-826b-78cd42616905 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:58.104076 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf0adc-79f4-425a-826b-78cd42616905 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:58.105657 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf0adc-79f4-425a-826b-78cd42616905 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:58.108583 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e5433-e2e9-4e5d-ac0d-c47e0cbef5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:58.109788 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e5433-e2e9-4e5d-ac0d-c47e0cbef5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:58.110275 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e5433-e2e9-4e5d-ac0d-c47e0cbef5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:58.110754 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e5433-e2e9-4e5d-ac0d-c47e0cbef5e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:58.122836 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e5433-e2e9-4e5d-ac0d-c47e0cbef5e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:58.123318 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 425/838] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:23:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:58.126232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf0adc-79f4-425a-826b-78cd42616905 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:58.128831 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-9b0e5433-e2e9-4e5d-ac0d-c47e0cbef5e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:58.128831 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:58.128831 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:58.128831 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:58.128831 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:58.128831 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:58.128831 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:58.128831 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:58.128831 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:58.128831 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:58.128831 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:58.128831 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:58.128831 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:58.128831 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:58.128831 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:58.128831 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:58.129748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:58.129748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:58.129748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:58.129748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:58.129748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:58.129748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:58.129748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:58.129748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:58.129748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:58.129748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:58.129748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:58.129748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:58.129748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:58.129748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:58.129748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:58.129748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:58.130581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:58.130581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:58.130581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:58.130581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:58.130581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:58.130581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:58.130581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:58.130581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:58.130581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:58.130581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:58.130581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:23:58.130581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:58.132703 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 414/839] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 12:23:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:23:58.140084 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e5433-e2e9-4e5d-ac0d-c47e0cbef5e2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:58.156619 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:58.178831 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:58.238889 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf0adc-79f4-425a-826b-78cd42616905 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:58.291401 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e5433-e2e9-4e5d-ac0d-c47e0cbef5e2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:58.293843 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2691e4-2944-458a-8c7b-18c1bd065d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:58.294681 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2691e4-2944-458a-8c7b-18c1bd065d46 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:58.295204 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2691e4-2944-458a-8c7b-18c1bd065d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:58.295706 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2691e4-2944-458a-8c7b-18c1bd065d46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:58.296662 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2691e4-2944-458a-8c7b-18c1bd065d46 None admin] RBAC: Adding query filter params (role.id=bffbe227cda1418cbb58394ddff6f861, scope.project.id=8e201e5f281a4977be503e22daf149d7, user.id=eb3f6deffb54467f8b923956ed2a6acf) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:58.297185 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2691e4-2944-458a-8c7b-18c1bd065d46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:58.298795 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2691e4-2944-458a-8c7b-18c1bd065d46 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:58.315274 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 426/840] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:23:58 2022] GET /identity/v3/role_assignments?user.id=eb3f6deffb54467f8b923956ed2a6acf&scope.project.id=8e201e5f281a4977be503e22daf149d7&role.id=bffbe227cda1418cbb58394ddff6f861 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:23:58.332547 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd46d44-34ae-4f81-a571-4e15d9fc3032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:58.333090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd46d44-34ae-4f81-a571-4e15d9fc3032 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:58.333372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd46d44-34ae-4f81-a571-4e15d9fc3032 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:58.333646 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd46d44-34ae-4f81-a571-4e15d9fc3032 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:58.334321 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd46d44-34ae-4f81-a571-4e15d9fc3032 None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:58.335266 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd46d44-34ae-4f81-a571-4e15d9fc3032 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:58.336699 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd46d44-34ae-4f81-a571-4e15d9fc3032 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:58.346645 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 415/841] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 12:23:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:58.352657 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2691e4-2944-458a-8c7b-18c1bd065d46 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:58.458579 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:58.469014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2691e4-2944-458a-8c7b-18c1bd065d46 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:58.505188 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f67d2-0fe4-4bb6-8d94-a127a8121e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:58.505832 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f67d2-0fe4-4bb6-8d94-a127a8121e4d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:58.506114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f67d2-0fe4-4bb6-8d94-a127a8121e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:58.506933 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f67d2-0fe4-4bb6-8d94-a127a8121e4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:58.517458 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f67d2-0fe4-4bb6-8d94-a127a8121e4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:58.518925 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f67d2-0fe4-4bb6-8d94-a127a8121e4d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:58.535415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-2a4f67d2-0fe4-4bb6-8d94-a127a8121e4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:58.535415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:58.535415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:58.535415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:58.535415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:58.535415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:58.535415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:58.535415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:58.535415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:58.535415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:58.535415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:58.535415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:58.535415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:58.535415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:58.535415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:58.535415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:58.536425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:58.536425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:58.536425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:58.536425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:58.536425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:58.536425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:58.536425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:58.536425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:58.536425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:58.536425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:58.536425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:58.536425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:58.536425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:58.536425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:58.536425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:58.536425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:58.536425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:58.538285 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:58.538285 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:58.538285 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:58.538285 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 427/842] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:58.543553 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd46d44-34ae-4f81-a571-4e15d9fc3032 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:58.635504 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:58.654609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd46d44-34ae-4f81-a571-4e15d9fc3032 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:58.691646 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc10c21-8126-46b3-b239-1187b9eee35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:58.692262 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc10c21-8126-46b3-b239-1187b9eee35d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:58.692546 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc10c21-8126-46b3-b239-1187b9eee35d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:58.692823 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc10c21-8126-46b3-b239-1187b9eee35d None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:58.693466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc10c21-8126-46b3-b239-1187b9eee35d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:58.693773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc10c21-8126-46b3-b239-1187b9eee35d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:58.695780 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc10c21-8126-46b3-b239-1187b9eee35d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:58.708160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 416/843] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:58.713687 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f67d2-0fe4-4bb6-8d94-a127a8121e4d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:58.818736 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f67d2-0fe4-4bb6-8d94-a127a8121e4d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:58.856131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0feee2c-7f56-4131-a9ad-4e2327db080a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:58.856777 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0feee2c-7f56-4131-a9ad-4e2327db080a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:58.857059 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0feee2c-7f56-4131-a9ad-4e2327db080a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:58.857441 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0feee2c-7f56-4131-a9ad-4e2327db080a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:58.865381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0feee2c-7f56-4131-a9ad-4e2327db080a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:58.866887 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0feee2c-7f56-4131-a9ad-4e2327db080a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:58.878819 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-a0feee2c-7f56-4131-a9ad-4e2327db080a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 12:23:58.878819 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:58.878819 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:58.878819 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:58.878819 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:58.878819 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:58.878819 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:58.878819 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:58.878819 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:58.878819 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:58.878819 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:58.878819 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:58.878819 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:23:58.878819 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:23:58.878819 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:23:58.878819 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:23:58.879842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:58.879842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:58.879842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:23:58.879842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:58.879842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:23:58.879842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:23:58.879842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:58.879842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:58.879842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:58.879842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:58.879842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:58.879842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:58.879842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:58.879842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:58.879842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:58.879842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:58.879842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:58.879842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:58.881431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:58.881431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:58.881431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:23:58.881431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:23:58.881431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:23:58.881431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:23:58.881431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:23:58.881431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:23:58.881431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 12:23:58.881431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:58.881431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 428/844] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 10 12:23:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:58.885921 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc10c21-8126-46b3-b239-1187b9eee35d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:58.995347 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc10c21-8126-46b3-b239-1187b9eee35d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:59.032329 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70adfc1e-f3ab-40cf-98db-10cb8c0816ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:59.032942 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70adfc1e-f3ab-40cf-98db-10cb8c0816ba None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:59.033226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70adfc1e-f3ab-40cf-98db-10cb8c0816ba None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:59.033503 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70adfc1e-f3ab-40cf-98db-10cb8c0816ba None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:59.034311 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70adfc1e-f3ab-40cf-98db-10cb8c0816ba None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:59.034621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70adfc1e-f3ab-40cf-98db-10cb8c0816ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:59.036098 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70adfc1e-f3ab-40cf-98db-10cb8c0816ba None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:59.055994 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 417/845] 10.222.0.49 () {56 vars in 1149 bytes} [Wed Aug 10 12:23:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:59.061663 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0feee2c-7f56-4131-a9ad-4e2327db080a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:59.064823 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:59.168828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0feee2c-7f56-4131-a9ad-4e2327db080a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:59.204048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3e047-32b1-40a3-8bfe-54e8837c7530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:59.204665 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3e047-32b1-40a3-8bfe-54e8837c7530 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:59.204949 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3e047-32b1-40a3-8bfe-54e8837c7530 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:59.205226 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3e047-32b1-40a3-8bfe-54e8837c7530 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:59.218343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8618b90-433f-48ef-bde0-fea56386c13a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:59.218882 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8618b90-433f-48ef-bde0-fea56386c13a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:59.218882 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8618b90-433f-48ef-bde0-fea56386c13a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:59.220895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 418/846] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:23:59.230099 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a319d-a3e1-433f-adcf-be40ffb1086f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:59.230383 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a319d-a3e1-433f-adcf-be40ffb1086f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:59.230660 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a319d-a3e1-433f-adcf-be40ffb1086f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:59.237975 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:59.251050 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-953a319d-a3e1-433f-adcf-be40ffb1086f None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:59.252807 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-953a319d-a3e1-433f-adcf-be40ffb1086f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:59.287510 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc3e047-32b1-40a3-8bfe-54e8837c7530 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:59.289032 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc3e047-32b1-40a3-8bfe-54e8837c7530 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:59.290866 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:59.338214 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 419/847] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:23:59.360408 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a319d-a3e1-433f-adcf-be40ffb1086f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:59.405229 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 429/848] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:23:59 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:23:59.411247 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3e047-32b1-40a3-8bfe-54e8837c7530 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:59.474241 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a319d-a3e1-433f-adcf-be40ffb1086f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:59.511735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6779a6b-825c-45a0-b86e-d0ff8f75528d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:59.512799 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6779a6b-825c-45a0-b86e-d0ff8f75528d None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:59.513085 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6779a6b-825c-45a0-b86e-d0ff8f75528d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:59.513522 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6779a6b-825c-45a0-b86e-d0ff8f75528d None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:59.514185 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6779a6b-825c-45a0-b86e-d0ff8f75528d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:59.515591 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6779a6b-825c-45a0-b86e-d0ff8f75528d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:59.519157 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:59.526550 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3e047-32b1-40a3-8bfe-54e8837c7530 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:59.528302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:59.562487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77ac2c-e2d3-4253-80f3-35cf4c8b802d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:59.563120 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77ac2c-e2d3-4253-80f3-35cf4c8b802d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:59.563455 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77ac2c-e2d3-4253-80f3-35cf4c8b802d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:59.563729 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77ac2c-e2d3-4253-80f3-35cf4c8b802d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:59.573543 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b77ac2c-e2d3-4253-80f3-35cf4c8b802d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:59.574979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b77ac2c-e2d3-4253-80f3-35cf4c8b802d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:59.589661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-9b77ac2c-e2d3-4253-80f3-35cf4c8b802d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:59.589661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:59.589661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:59.589661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:59.589661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:59.589661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:59.589661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:59.589661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:59.589661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:59.589661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:59.589661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:59.589661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:59.589661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:23:59.589661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:23:59.589661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:23:59.589661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:23:59.590825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:59.590825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:59.590825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:59.590825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:59.590825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:59.590825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:59.590825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:59.590825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:59.590825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:59.590825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:59.590825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:59.590825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:59.590825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:59.590825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:59.590825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:59.590825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:59.590825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:23:59.591798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:23:59.591798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:23:59.591798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:23:59.591798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 430/849] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:23:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:23:59.592273 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0ce26-6d15-462a-a85d-814c0284a204 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:59.592557 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0ce26-6d15-462a-a85d-814c0284a204 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:59.592834 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0ce26-6d15-462a-a85d-814c0284a204 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:59.595204 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 431/850] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:59.596311 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0ce26-6d15-462a-a85d-814c0284a204 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:59.611221 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 420/851] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 10 12:23:59 2022] POST /identity/v3/services => generated 251 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:23:59.611221 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5a784-a77a-4392-9469-93c0c2052c02 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:59.611221 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5a784-a77a-4392-9469-93c0c2052c02 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:59.611221 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5a784-a77a-4392-9469-93c0c2052c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:59.633227 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a6e5a784-a77a-4392-9469-93c0c2052c02 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:59.635156 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a6e5a784-a77a-4392-9469-93c0c2052c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:59.743224 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0ce26-6d15-462a-a85d-814c0284a204 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:59.751372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 421/852] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:59 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:23:59.773325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e5a784-a77a-4392-9469-93c0c2052c02 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:59.793261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10b420-90ce-492e-96a3-38564070b741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:59.794289 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10b420-90ce-492e-96a3-38564070b741 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:59.795026 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10b420-90ce-492e-96a3-38564070b741 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:59.795346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10b420-90ce-492e-96a3-38564070b741 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:59.795996 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef10b420-90ce-492e-96a3-38564070b741 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:23:59.796306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef10b420-90ce-492e-96a3-38564070b741 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:59.797670 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef10b420-90ce-492e-96a3-38564070b741 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:59.811009 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 432/853] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:23:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:23:59.816315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10b420-90ce-492e-96a3-38564070b741 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:59.830090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:59.914783 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e5a784-a77a-4392-9469-93c0c2052c02 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:59.929453 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10b420-90ce-492e-96a3-38564070b741 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:59.947738 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:59.962083 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c3723-9d2d-4a32-b750-1c3f67b65922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:59.962725 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c3723-9d2d-4a32-b750-1c3f67b65922 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:59.963001 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c3723-9d2d-4a32-b750-1c3f67b65922 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:59.963305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c3723-9d2d-4a32-b750-1c3f67b65922 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:59.966084 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f16b5-3a63-44ef-902d-90b308889fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:59.966853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f16b5-3a63-44ef-902d-90b308889fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:59.967213 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f16b5-3a63-44ef-902d-90b308889fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:59.967555 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f16b5-3a63-44ef-902d-90b308889fb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:59.974702 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09c3723-9d2d-4a32-b750-1c3f67b65922 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:23:59.976616 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81f16b5-3a63-44ef-902d-90b308889fb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:59.978188 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81f16b5-3a63-44ef-902d-90b308889fb5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:59.980853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f09c3723-9d2d-4a32-b750-1c3f67b65922 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:23:59.980853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:59.980853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:59.980853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:59.980853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:59.980853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:59.980853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:59.980853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:59.980853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:59.980853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:59.980853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:59.980853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:59.980853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:23:59.980853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:23:59.980853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:23:59.980853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:23:59.981925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:23:59.981925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:23:59.981925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:23:59.981925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:23:59.981925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:59.981925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:59.981925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:59.981925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:59.981925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:59.981925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:59.981925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:59.981925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:59.981925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:59.981925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:59.981925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:59.981925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:59.982791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:59.982791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:59.982791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:59.982791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:23:59.982791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:23:59.982791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:23:59.982791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:23:59.982791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:23:59.982791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:23:59.982791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:23:59.982791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:23:59.982791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:23:59.982791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 422/854] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:23:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:23:59.990337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c3723-9d2d-4a32-b750-1c3f67b65922 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:59.998716 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-d81f16b5-3a63-44ef-902d-90b308889fb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:23:59.998716 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:23:59.998716 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:23:59.998716 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:23:59.998716 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:23:59.998716 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:23:59.998716 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:23:59.998716 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:23:59.998716 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:23:59.998716 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:23:59.998716 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:23:59.998716 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:23:59.998716 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:23:59.998716 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:23:59.998716 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:23:59.999847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:23:59.999847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:23:59.999847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:23:59.999847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:23:59.999847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:23:59.999847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:23:59.999847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:23:59.999847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:23:59.999847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:23:59.999847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:23:59.999847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:23:59.999847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:23:59.999847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:23:59.999847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:23:59.999847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:23:59.999847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:23:59.999847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:00.000764 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:24:00.000764 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:24:00.000764 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:24:00.000764 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:24:00.000764 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:24:00.000764 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:24:00.000764 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:24:00.000764 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:00.000764 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 433/855] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:23:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:24:00.007592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f16b5-3a63-44ef-902d-90b308889fb5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:00.102183 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:00.106069 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c3723-9d2d-4a32-b750-1c3f67b65922 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:00.143594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68de34-ab5e-4a40-9fe9-ef7fed325e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:00.144352 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68de34-ab5e-4a40-9fe9-ef7fed325e3d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:00.144734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68de34-ab5e-4a40-9fe9-ef7fed325e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:00.145140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68de34-ab5e-4a40-9fe9-ef7fed325e3d None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:00.145636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f16b5-3a63-44ef-902d-90b308889fb5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:00.146036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68de34-ab5e-4a40-9fe9-ef7fed325e3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:00.146435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68de34-ab5e-4a40-9fe9-ef7fed325e3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:00.148619 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68de34-ab5e-4a40-9fe9-ef7fed325e3d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:00.162489 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 423/856] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:23:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:00.164195 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68de34-ab5e-4a40-9fe9-ef7fed325e3d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:00.191309 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efdd446-29bf-4860-9c01-6053f92c9823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:00.192066 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdd446-29bf-4860-9c01-6053f92c9823 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:00.192702 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdd446-29bf-4860-9c01-6053f92c9823 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:00.192853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdd446-29bf-4860-9c01-6053f92c9823 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:00.193690 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efdd446-29bf-4860-9c01-6053f92c9823 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:00.194167 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efdd446-29bf-4860-9c01-6053f92c9823 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:00.195811 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efdd446-29bf-4860-9c01-6053f92c9823 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:00.213769 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 434/857] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:24:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:00.216664 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efdd446-29bf-4860-9c01-6053f92c9823 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:00.221803 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:00.303691 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68de34-ab5e-4a40-9fe9-ef7fed325e3d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:00.327732 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efdd446-29bf-4860-9c01-6053f92c9823 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:00.340946 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06661762-6d71-45c6-a349-c86e1fd85b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:00.341584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06661762-6d71-45c6-a349-c86e1fd85b02 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:00.341864 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06661762-6d71-45c6-a349-c86e1fd85b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:00.342139 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06661762-6d71-45c6-a349-c86e1fd85b02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:00.354093 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06661762-6d71-45c6-a349-c86e1fd85b02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:00.355585 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06661762-6d71-45c6-a349-c86e1fd85b02 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:00.359438 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:00.365636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc204c-d699-4bb3-9b47-2a08c610bf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:00.367137 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc204c-d699-4bb3-9b47-2a08c610bf48 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:00.367501 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc204c-d699-4bb3-9b47-2a08c610bf48 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:00.370575 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc204c-d699-4bb3-9b47-2a08c610bf48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:00.370575 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acc204c-d699-4bb3-9b47-2a08c610bf48 None admin] RBAC: Adding query filter params (role.id=a9bbb011afca4176a3def9356cb5f165, scope.project.id=8e201e5f281a4977be503e22daf149d7, user.id=e42059e9af9842548ea08dcd39ff0b6e) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:00.370575 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acc204c-d699-4bb3-9b47-2a08c610bf48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:00.370575 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acc204c-d699-4bb3-9b47-2a08c610bf48 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:00.372624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-06661762-6d71-45c6-a349-c86e1fd85b02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:00.372624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:00.372624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:00.372624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:00.372624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:00.372624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:00.372624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:00.372624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:00.372624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:00.372624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:00.372624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:00.372624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:00.372624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:24:00.372624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:24:00.372624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:24:00.372624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:24:00.374470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:00.374470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:00.374470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:00.374470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:00.374470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:00.374470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:00.374470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:00.374470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:00.374470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:00.374470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:00.374470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:00.374470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:00.374470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:00.374470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:00.374470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:00.374470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:00.374470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:24:00.376719 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:24:00.376719 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:00.376719 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:00.378366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 435/859] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:24:00 2022] GET /identity/v3/role_assignments?user.id=e42059e9af9842548ea08dcd39ff0b6e&scope.project.id=8e201e5f281a4977be503e22daf149d7&role.id=a9bbb011afca4176a3def9356cb5f165 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:00.378527 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 424/859] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:24:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:00.383204 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc204c-d699-4bb3-9b47-2a08c610bf48 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:00.461379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b5e31-a0f0-4475-9d63-287d87132636 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:00.462586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b5e31-a0f0-4475-9d63-287d87132636 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:00.462586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b5e31-a0f0-4475-9d63-287d87132636 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:00.464807 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 425/860] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:24:00.472165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c27218-d138-473b-9978-1dd4f75bc86b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:00.472565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c27218-d138-473b-9978-1dd4f75bc86b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:00.472918 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c27218-d138-473b-9978-1dd4f75bc86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:00.493706 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-34c27218-d138-473b-9978-1dd4f75bc86b None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:00.495069 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-34c27218-d138-473b-9978-1dd4f75bc86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:00.497221 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc204c-d699-4bb3-9b47-2a08c610bf48 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:00.541991 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8679a39-1bde-48c7-92bc-ead91a3dbfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:00.542583 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8679a39-1bde-48c7-92bc-ead91a3dbfac None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:00.542866 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8679a39-1bde-48c7-92bc-ead91a3dbfac None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:00.543138 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8679a39-1bde-48c7-92bc-ead91a3dbfac None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:00.543810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8679a39-1bde-48c7-92bc-ead91a3dbfac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:00.544110 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8679a39-1bde-48c7-92bc-ead91a3dbfac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:00.545472 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8679a39-1bde-48c7-92bc-ead91a3dbfac None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:00.560673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 436/861] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:00.566514 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8679a39-1bde-48c7-92bc-ead91a3dbfac None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:00.585166 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 426/862] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:00 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:00.607924 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c27218-d138-473b-9978-1dd4f75bc86b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:00.637053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:00.648161 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:00.691449 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8679a39-1bde-48c7-92bc-ead91a3dbfac None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:00.739007 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634f459-6b51-456c-a6cf-e7b9db35fd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:00.740424 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634f459-6b51-456c-a6cf-e7b9db35fd5f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:00.741212 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634f459-6b51-456c-a6cf-e7b9db35fd5f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:00.741697 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634f459-6b51-456c-a6cf-e7b9db35fd5f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:00.742785 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c27218-d138-473b-9978-1dd4f75bc86b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:00.751379 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3634f459-6b51-456c-a6cf-e7b9db35fd5f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:00.754089 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3634f459-6b51-456c-a6cf-e7b9db35fd5f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:00.771780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-3634f459-6b51-456c-a6cf-e7b9db35fd5f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 12:24:00.771780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:00.771780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:00.771780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:00.771780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:00.771780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:00.771780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:00.771780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:00.771780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:00.771780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:00.771780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:00.771780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:00.771780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:24:00.771780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:24:00.771780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:24:00.771780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:24:00.772931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:00.772931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:00.772931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:24:00.772931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:24:00.772931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:24:00.772931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:24:00.772931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:00.772931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:00.772931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:00.772931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:00.772931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:00.772931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:00.772931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:00.772931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:00.772931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:00.772931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:00.772931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:00.772931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:00.773869 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:00.773869 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:00.773869 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:24:00.773869 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:24:00.773869 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:24:00.773869 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:24:00.773869 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:24:00.773869 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:24:00.773869 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 12:24:00.773869 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:00.777290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 437/863] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:24:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:00.782831 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634f459-6b51-456c-a6cf-e7b9db35fd5f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:00.789443 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099965e-596b-4e93-b753-ae41086df8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:00.790168 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099965e-596b-4e93-b753-ae41086df8cd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:00.790522 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099965e-596b-4e93-b753-ae41086df8cd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:00.790865 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099965e-596b-4e93-b753-ae41086df8cd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:00.791565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c099965e-596b-4e93-b753-ae41086df8cd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:00.793079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c099965e-596b-4e93-b753-ae41086df8cd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:00.802171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-c099965e-596b-4e93-b753-ae41086df8cd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 12:24:00.802171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:00.802171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:24:00.802171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:24:00.802171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:24:00.802171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:24:00.802171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:24:00.802171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:00.802171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 12:24:00.802171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:00.802171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:24:00.802171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:00.802171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:00.802171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:00.802171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:00.802171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:00.803113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:00.803113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:00.803113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:00.803113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:00.803113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:00.803113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:00.803113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:00.803113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:24:00.803113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:24:00.803113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:24:00.803113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:24:00.803113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:00.803113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:00.803113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:00.803113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:00.803113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:00.803113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:00.804771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:00.804771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:00.804771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:00.804771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:00.804771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:00.804771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:00.804771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:00.804771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:00.804771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:00.804771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:00.804771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:24:00.804771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:00.804771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 12:24:00.804771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:00.804771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 427/864] 10.222.0.49 () {56 vars in 1132 bytes} [Wed Aug 10 12:24:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:00.809608 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099965e-596b-4e93-b753-ae41086df8cd None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:00.919486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634f459-6b51-456c-a6cf-e7b9db35fd5f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:00.937526 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:00.938498 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:00.949616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099965e-596b-4e93-b753-ae41086df8cd None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:00.964366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0192dbe-62e8-43ff-9df0-476d39e3f6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:00.965681 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0192dbe-62e8-43ff-9df0-476d39e3f6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:00.966216 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0192dbe-62e8-43ff-9df0-476d39e3f6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:00.966637 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0192dbe-62e8-43ff-9df0-476d39e3f6e8 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:00.967691 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0192dbe-62e8-43ff-9df0-476d39e3f6e8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:00.968534 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0192dbe-62e8-43ff-9df0-476d39e3f6e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:00.970661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0192dbe-62e8-43ff-9df0-476d39e3f6e8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:00.991344 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c51e93-f2e7-4979-b0e3-997d71b9d6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:00.992104 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c51e93-f2e7-4979-b0e3-997d71b9d6b2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:00.992476 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c51e93-f2e7-4979-b0e3-997d71b9d6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:00.992656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c51e93-f2e7-4979-b0e3-997d71b9d6b2 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:00.998624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c51e93-f2e7-4979-b0e3-997d71b9d6b2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:00.998624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c51e93-f2e7-4979-b0e3-997d71b9d6b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:00.998624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c51e93-f2e7-4979-b0e3-997d71b9d6b2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:00.998900 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 438/865] 10.222.0.49 () {56 vars in 1135 bytes} [Wed Aug 10 12:24:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:01.001065 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0192dbe-62e8-43ff-9df0-476d39e3f6e8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:01.004971 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 428/866] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 10 12:24:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:01.011243 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c51e93-f2e7-4979-b0e3-997d71b9d6b2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:01.125465 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c51e93-f2e7-4979-b0e3-997d71b9d6b2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:01.148966 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0192dbe-62e8-43ff-9df0-476d39e3f6e8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:01.163713 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f92426-f8e9-489a-9f6a-955d0b799650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:01.164502 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f92426-f8e9-489a-9f6a-955d0b799650 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:01.164930 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f92426-f8e9-489a-9f6a-955d0b799650 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:01.165305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f92426-f8e9-489a-9f6a-955d0b799650 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:01.166037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f92426-f8e9-489a-9f6a-955d0b799650 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:01.166688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f92426-f8e9-489a-9f6a-955d0b799650 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:01.167976 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f92426-f8e9-489a-9f6a-955d0b799650 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:01.179222 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 429/867] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 10 12:24:01 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:01.199942 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a1f81-a3ec-4838-9a07-377e5411a600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:01.200571 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a1f81-a3ec-4838-9a07-377e5411a600 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:01.200851 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a1f81-a3ec-4838-9a07-377e5411a600 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:01.201122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a1f81-a3ec-4838-9a07-377e5411a600 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:01.249868 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:01.296403 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a1f81-a3ec-4838-9a07-377e5411a600 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:01.298292 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a1f81-a3ec-4838-9a07-377e5411a600 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:01.406683 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 439/868] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:24:01 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 12:24:01.412671 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f92426-f8e9-489a-9f6a-955d0b799650 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:01.438024 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:01.519937 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f92426-f8e9-489a-9f6a-955d0b799650 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:01.559375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ffe17-d4e5-438d-bd31-7d4852025bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:01.560006 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ffe17-d4e5-438d-bd31-7d4852025bfb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:01.560283 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ffe17-d4e5-438d-bd31-7d4852025bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:01.560556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ffe17-d4e5-438d-bd31-7d4852025bfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:01.571029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ffe17-d4e5-438d-bd31-7d4852025bfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:01.572849 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ffe17-d4e5-438d-bd31-7d4852025bfb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:01.587644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b29ffe17-d4e5-438d-bd31-7d4852025bfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:01.587644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:01.587644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:01.587644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:01.587644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:01.587644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:01.587644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:01.587644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:01.587644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:01.587644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:01.587644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:01.587644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:01.587644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:24:01.587644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:24:01.587644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:24:01.587644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:24:01.588642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:01.588642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:01.588642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:01.588642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:01.588642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:01.588642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:01.588642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:01.588642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:01.588642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:01.588642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:01.588642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:01.588642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:01.588642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:01.588642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:01.588642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:01.588642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:01.588642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:24:01.589560 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:24:01.589560 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:01.589560 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:01.589560 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 430/869] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:24:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:01.595900 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a1f81-a3ec-4838-9a07-377e5411a600 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:01.680333 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:01.714053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a1f81-a3ec-4838-9a07-377e5411a600 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:01.750251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3e0ba-0acc-46ac-9d6c-b14771f7e01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:01.750863 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3e0ba-0acc-46ac-9d6c-b14771f7e01b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:01.751143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3e0ba-0acc-46ac-9d6c-b14771f7e01b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:01.751444 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3e0ba-0acc-46ac-9d6c-b14771f7e01b None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:01.752081 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3e0ba-0acc-46ac-9d6c-b14771f7e01b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:01.752387 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3e0ba-0acc-46ac-9d6c-b14771f7e01b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:01.753743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3e0ba-0acc-46ac-9d6c-b14771f7e01b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:01.759484 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f1059-52bd-4783-b617-b5eb5755a8d5 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:01.759769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f1059-52bd-4783-b617-b5eb5755a8d5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:01.760044 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f1059-52bd-4783-b617-b5eb5755a8d5 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:01.761683 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 431/870] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:01.765848 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 440/871] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:24:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:01.766477 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08dc11-b8b1-4f3b-a77f-e24e9ba67389 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:01.766779 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08dc11-b8b1-4f3b-a77f-e24e9ba67389 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:01.767092 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08dc11-b8b1-4f3b-a77f-e24e9ba67389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:01.771481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3e0ba-0acc-46ac-9d6c-b14771f7e01b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:01.785749 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-fe08dc11-b8b1-4f3b-a77f-e24e9ba67389 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:01.786927 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fe08dc11-b8b1-4f3b-a77f-e24e9ba67389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:01.859428 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 432/872] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:01 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:24:01.874211 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3e0ba-0acc-46ac-9d6c-b14771f7e01b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:01.878256 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe08dc11-b8b1-4f3b-a77f-e24e9ba67389 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:01.895525 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:01.906008 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:01.912102 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b57d7-79bd-43cb-acf7-246f458b036a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:01.913417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b57d7-79bd-43cb-acf7-246f458b036a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:01.913894 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b57d7-79bd-43cb-acf7-246f458b036a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:01.914373 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b57d7-79bd-43cb-acf7-246f458b036a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:01.921708 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b57d7-79bd-43cb-acf7-246f458b036a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:01.923407 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b57d7-79bd-43cb-acf7-246f458b036a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:01.935927 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-960b57d7-79bd-43cb-acf7-246f458b036a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:24:01.935927 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:01.935927 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:01.935927 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:01.935927 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:01.935927 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:01.935927 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:01.935927 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:01.935927 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:01.935927 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:01.935927 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:01.935927 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:01.935927 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:24:01.935927 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:24:01.935927 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:24:01.936821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:24:01.936821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:01.936821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:01.936821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:01.936821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:01.936821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:01.936821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:01.936821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:01.936821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:01.936821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:01.936821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:01.936821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:01.936821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:01.936821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:01.936821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:01.936821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:01.936821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:01.937770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:24:01.937770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:24:01.937770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:24:01.937770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:24:01.937770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:24:01.937770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:24:01.937770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:24:01.937770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:01.940186 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 441/873] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:24:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:01.945523 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b57d7-79bd-43cb-acf7-246f458b036a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:01.993226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe08dc11-b8b1-4f3b-a77f-e24e9ba67389 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:02.035193 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d81c2e-b585-43e3-8301-a41615ac71cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:02.035835 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d81c2e-b585-43e3-8301-a41615ac71cb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:02.036110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d81c2e-b585-43e3-8301-a41615ac71cb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:02.036380 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d81c2e-b585-43e3-8301-a41615ac71cb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:02.037032 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d81c2e-b585-43e3-8301-a41615ac71cb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:02.038684 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d81c2e-b585-43e3-8301-a41615ac71cb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:02.047952 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d1d81c2e-b585-43e3-8301-a41615ac71cb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 12:24:02.047952 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:02.047952 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:24:02.047952 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:24:02.047952 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:24:02.047952 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:24:02.047952 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:24:02.047952 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:02.047952 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 12:24:02.047952 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:02.047952 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:24:02.047952 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:02.047952 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:02.047952 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:02.047952 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:02.047952 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:02.047952 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:02.049152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:02.049152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:02.049152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:02.049152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:02.049152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:02.049152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:02.049152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:24:02.049152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:24:02.049152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:24:02.049152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:24:02.049152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:02.049152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:02.049152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:02.049152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:02.049152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:02.049152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:02.050171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:02.050171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:02.050171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:02.050171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:02.050171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:02.050171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:02.050171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:02.050171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:02.050171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:02.050171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:02.050171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:24:02.050171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:02.050171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 12:24:02.050171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:02.054670 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 433/874] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 12:24:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:24:02.055643 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d81c2e-b585-43e3-8301-a41615ac71cb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:02.072716 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b57d7-79bd-43cb-acf7-246f458b036a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:02.108660 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d1e22-8e3c-418f-a87b-ef87248dcd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:02.109423 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d1e22-8e3c-418f-a87b-ef87248dcd88 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:02.109896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d1e22-8e3c-418f-a87b-ef87248dcd88 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:02.110274 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d1e22-8e3c-418f-a87b-ef87248dcd88 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:02.111079 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d1e22-8e3c-418f-a87b-ef87248dcd88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:02.111518 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d1e22-8e3c-418f-a87b-ef87248dcd88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:02.113239 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d1e22-8e3c-418f-a87b-ef87248dcd88 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:02.123289 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:02.128490 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 442/875] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:24:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:02.134883 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d1e22-8e3c-418f-a87b-ef87248dcd88 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:02.159861 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d81c2e-b585-43e3-8301-a41615ac71cb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:02.190843 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:02.194974 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75a6bc-6007-42b6-bca2-090147363c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:02.195653 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75a6bc-6007-42b6-bca2-090147363c37 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:02.195882 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75a6bc-6007-42b6-bca2-090147363c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:02.196151 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75a6bc-6007-42b6-bca2-090147363c37 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:02.197016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75a6bc-6007-42b6-bca2-090147363c37 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:02.197133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75a6bc-6007-42b6-bca2-090147363c37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:02.198470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75a6bc-6007-42b6-bca2-090147363c37 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:02.208086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 434/876] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 12:24:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:02.209043 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75a6bc-6007-42b6-bca2-090147363c37 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:02.243580 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d1e22-8e3c-418f-a87b-ef87248dcd88 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:02.290177 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfdc8c-00b7-4c41-80b5-83fb5fc3cf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:02.291023 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfdc8c-00b7-4c41-80b5-83fb5fc3cf59 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:02.291478 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfdc8c-00b7-4c41-80b5-83fb5fc3cf59 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:02.291897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfdc8c-00b7-4c41-80b5-83fb5fc3cf59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:02.292765 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cfdc8c-00b7-4c41-80b5-83fb5fc3cf59 None admin] RBAC: Adding query filter params (role.id=bffbe227cda1418cbb58394ddff6f861, scope.project.id=8e201e5f281a4977be503e22daf149d7, user.id=59395bfa505c4540b3957deee78eb698) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:02.293243 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cfdc8c-00b7-4c41-80b5-83fb5fc3cf59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:02.294796 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cfdc8c-00b7-4c41-80b5-83fb5fc3cf59 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:02.307449 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 443/877] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:24:02 2022] GET /identity/v3/role_assignments?user.id=59395bfa505c4540b3957deee78eb698&scope.project.id=8e201e5f281a4977be503e22daf149d7&role.id=bffbe227cda1418cbb58394ddff6f861 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:02.313999 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75a6bc-6007-42b6-bca2-090147363c37 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:02.351493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:02.355795 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b404d4-dd84-41bf-b8d9-2d3091261f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:02.356541 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b404d4-dd84-41bf-b8d9-2d3091261f04 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:02.356893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b404d4-dd84-41bf-b8d9-2d3091261f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:02.357238 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b404d4-dd84-41bf-b8d9-2d3091261f04 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:02.357949 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b404d4-dd84-41bf-b8d9-2d3091261f04 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:02.358324 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b404d4-dd84-41bf-b8d9-2d3091261f04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:02.359925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b404d4-dd84-41bf-b8d9-2d3091261f04 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:02.366775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 435/878] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 12:24:02 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:02.373399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfdc8c-00b7-4c41-80b5-83fb5fc3cf59 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:02.493709 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfdc8c-00b7-4c41-80b5-83fb5fc3cf59 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:02.511390 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:02.529539 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc86df-705a-45f0-9a3e-090de95fbcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:02.530151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc86df-705a-45f0-9a3e-090de95fbcd4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:02.530431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc86df-705a-45f0-9a3e-090de95fbcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:02.530704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc86df-705a-45f0-9a3e-090de95fbcd4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:02.531431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc86df-705a-45f0-9a3e-090de95fbcd4 None admin] RBAC: Adding query filter params (interface=public, service_id=46cdcc20e46c4ec5bd813167e3be4255, region_id=RegionOne) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:02.531801 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc86df-705a-45f0-9a3e-090de95fbcd4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:02.533168 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc86df-705a-45f0-9a3e-090de95fbcd4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:02.541378 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 444/879] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Aug 10 12:24:02 2022] GET /identity/v3/endpoints?service_id=46cdcc20e46c4ec5bd813167e3be4255&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:02.547135 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b404d4-dd84-41bf-b8d9-2d3091261f04 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:02.623016 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266829b-5cd8-49d4-a1f6-1412ff48bf67 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:02.623359 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266829b-5cd8-49d4-a1f6-1412ff48bf67 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:02.623605 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266829b-5cd8-49d4-a1f6-1412ff48bf67 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:02.625196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 445/880] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:02.629998 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e86f5-265f-4a75-947e-540b48d27b45 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:02.630280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e86f5-265f-4a75-947e-540b48d27b45 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:02.630558 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e86f5-265f-4a75-947e-540b48d27b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:02.654975 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b404d4-dd84-41bf-b8d9-2d3091261f04 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:02.655662 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b87e86f5-265f-4a75-947e-540b48d27b45 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:02.657117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b87e86f5-265f-4a75-947e-540b48d27b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:02.692448 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc58769-dd4a-45c1-8a36-d414856fe6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:02.693220 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc58769-dd4a-45c1-8a36-d414856fe6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:02.693571 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc58769-dd4a-45c1-8a36-d414856fe6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:02.694064 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc58769-dd4a-45c1-8a36-d414856fe6c9 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:02.695214 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc58769-dd4a-45c1-8a36-d414856fe6c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:02.696649 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc58769-dd4a-45c1-8a36-d414856fe6c9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:02.702662 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 436/881] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Aug 10 12:24:02 2022] GET /identity/v3/services => generated 1011 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:24:02.748188 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 446/882] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:02 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:24:02.775411 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc58769-dd4a-45c1-8a36-d414856fe6c9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:02.848141 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:02.900435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc58769-dd4a-45c1-8a36-d414856fe6c9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:02.937168 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60189c7-a321-498f-be45-f22a3d01bb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:02.938102 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60189c7-a321-498f-be45-f22a3d01bb99 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:02.938620 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60189c7-a321-498f-be45-f22a3d01bb99 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:02.939074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60189c7-a321-498f-be45-f22a3d01bb99 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:02.950799 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60189c7-a321-498f-be45-f22a3d01bb99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:24:02.956532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a60189c7-a321-498f-be45-f22a3d01bb99 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 12:24:02.956532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:02.956532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:02.956532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:02.956532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:02.956532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:02.956532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:02.956532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:02.956532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:02.956532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:02.956532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:02.956532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:02.956532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:24:02.956532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:24:02.956532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:24:02.956532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:24:02.963944 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:24:02.963944 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:24:02.963944 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:24:02.963944 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:24:02.963944 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:02.963944 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:02.963944 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:02.963944 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:02.963944 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:02.963944 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:02.963944 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:02.963944 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:02.963944 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:02.963944 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:02.963944 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:02.963944 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:02.964940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:02.964940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:02.964940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:02.964940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:02.964940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:24:02.964940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:24:02.964940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:24:02.964940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:24:02.964940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:24:02.964940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:24:02.964940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 12:24:02.964940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:02.964940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 437/883] 10.222.0.49 () {56 vars in 1120 bytes} [Wed Aug 10 12:24:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:02.966619 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e86f5-265f-4a75-947e-540b48d27b45 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:03.075097 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e86f5-265f-4a75-947e-540b48d27b45 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:03.086873 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:03.110773 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d716d4-9bf6-438b-8156-52d89ae5d5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:03.111421 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d716d4-9bf6-438b-8156-52d89ae5d5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:03.111703 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d716d4-9bf6-438b-8156-52d89ae5d5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:03.111978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d716d4-9bf6-438b-8156-52d89ae5d5d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:03.112699 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d716d4-9bf6-438b-8156-52d89ae5d5d6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:03.113003 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d716d4-9bf6-438b-8156-52d89ae5d5d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:03.114456 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d716d4-9bf6-438b-8156-52d89ae5d5d6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:03.124438 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 447/884] 10.222.0.49 () {56 vars in 1115 bytes} [Wed Aug 10 12:24:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:03.129680 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60189c7-a321-498f-be45-f22a3d01bb99 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:03.243015 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60189c7-a321-498f-be45-f22a3d01bb99 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:03.267961 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:03.284152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4430580b-f60a-4b57-a80c-9128235c1750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:03.284964 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430580b-f60a-4b57-a80c-9128235c1750 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:03.285404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430580b-f60a-4b57-a80c-9128235c1750 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:03.285816 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430580b-f60a-4b57-a80c-9128235c1750 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:03.298818 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4430580b-f60a-4b57-a80c-9128235c1750 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:03.300549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4430580b-f60a-4b57-a80c-9128235c1750 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:03.316277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-4430580b-f60a-4b57-a80c-9128235c1750 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:03.316277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:03.316277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:03.316277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:03.316277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:03.316277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:03.316277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:03.316277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:03.316277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:03.316277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:03.316277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:03.316277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:03.316277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:24:03.316277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:24:03.316277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:24:03.316277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:24:03.317448 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:03.317448 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:03.317448 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:03.317448 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:03.317448 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:03.317448 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:03.317448 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:03.317448 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:03.317448 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:03.317448 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:03.317448 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:03.317448 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:03.317448 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:03.317448 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:03.317448 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:03.317448 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:03.317448 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:24:03.318611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:24:03.318611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:03.318611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:03.321464 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 438/885] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:24:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:03.327361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d716d4-9bf6-438b-8156-52d89ae5d5d6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:03.355444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7522b3-f1f6-402d-b347-16d36933cb5e None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:03.355989 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7522b3-f1f6-402d-b347-16d36933cb5e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:03.356431 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7522b3-f1f6-402d-b347-16d36933cb5e None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:03.358196 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 439/886] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:24:03.365182 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bd2b0-7dc4-44ed-9005-e6aefe142af9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:03.365899 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bd2b0-7dc4-44ed-9005-e6aefe142af9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:03.367043 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bd2b0-7dc4-44ed-9005-e6aefe142af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:03.392463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ee4bd2b0-7dc4-44ed-9005-e6aefe142af9 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:03.394395 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ee4bd2b0-7dc4-44ed-9005-e6aefe142af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:03.445907 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d716d4-9bf6-438b-8156-52d89ae5d5d6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:03.483656 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb88da-d995-462d-9eb3-3e86e1dc0b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:03.484362 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb88da-d995-462d-9eb3-3e86e1dc0b47 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:03.484715 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb88da-d995-462d-9eb3-3e86e1dc0b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:03.485044 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb88da-d995-462d-9eb3-3e86e1dc0b47 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:03.485958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffb88da-d995-462d-9eb3-3e86e1dc0b47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:03.486075 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffb88da-d995-462d-9eb3-3e86e1dc0b47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:03.487547 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffb88da-d995-462d-9eb3-3e86e1dc0b47 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:03.499493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 440/887] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:03 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:03.501556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 448/888] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:03.507647 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4bd2b0-7dc4-44ed-9005-e6aefe142af9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:03.521421 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb88da-d995-462d-9eb3-3e86e1dc0b47 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:03.555651 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:03.622848 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:03.627335 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4bd2b0-7dc4-44ed-9005-e6aefe142af9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:03.634357 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb88da-d995-462d-9eb3-3e86e1dc0b47 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:03.672072 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe43ff6-a0dd-499e-8327-49cc7d0a068a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:03.672423 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183e222-6692-4437-aa9e-383983d0d0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:03.673062 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183e222-6692-4437-aa9e-383983d0d0ee None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:03.673253 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe43ff6-a0dd-499e-8327-49cc7d0a068a None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:03.673361 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183e222-6692-4437-aa9e-383983d0d0ee None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:03.673606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183e222-6692-4437-aa9e-383983d0d0ee None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:03.673906 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe43ff6-a0dd-499e-8327-49cc7d0a068a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:03.674335 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe43ff6-a0dd-499e-8327-49cc7d0a068a None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:03.675115 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe43ff6-a0dd-499e-8327-49cc7d0a068a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:03.676844 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe43ff6-a0dd-499e-8327-49cc7d0a068a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:03.681957 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b183e222-6692-4437-aa9e-383983d0d0ee None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:03.684374 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b183e222-6692-4437-aa9e-383983d0d0ee None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:03.697740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b183e222-6692-4437-aa9e-383983d0d0ee None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 12:24:03.697740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:03.697740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:03.697740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:03.697740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:03.697740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:03.697740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:03.697740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:03.697740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:03.697740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:03.697740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:03.697740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:03.697740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:24:03.697740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:24:03.697740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:24:03.697740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:24:03.698721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:03.698721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:03.698721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:24:03.698721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:24:03.698721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:24:03.698721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:24:03.698721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:03.698721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:03.698721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:03.698721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:03.698721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:03.698721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:03.698721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:03.698721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:03.698721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:03.698721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:03.698721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:03.698721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:03.699720 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:03.699720 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:03.699720 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:24:03.699720 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:24:03.699720 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:24:03.699720 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:24:03.699720 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:24:03.699720 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:24:03.699720 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 12:24:03.699720 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:03.699720 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 441/889] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 10 12:24:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:03.706114 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183e222-6692-4437-aa9e-383983d0d0ee None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:03.778812 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 449/890] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 10 12:24:03 2022] POST /identity/v3/services => generated 269 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:24:03.824697 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183e222-6692-4437-aa9e-383983d0d0ee None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:03.871976 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd3a05-b092-4ba3-804a-a6a425668fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:03.872592 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd3a05-b092-4ba3-804a-a6a425668fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:03.872940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd3a05-b092-4ba3-804a-a6a425668fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:03.873159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd3a05-b092-4ba3-804a-a6a425668fa4 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:03.873894 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd3a05-b092-4ba3-804a-a6a425668fa4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:03.874203 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd3a05-b092-4ba3-804a-a6a425668fa4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:03.875812 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd3a05-b092-4ba3-804a-a6a425668fa4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:03.900046 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 442/891] 10.222.0.49 () {56 vars in 1149 bytes} [Wed Aug 10 12:24:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:03.905582 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe43ff6-a0dd-499e-8327-49cc7d0a068a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:04.013255 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:04.023835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe43ff6-a0dd-499e-8327-49cc7d0a068a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:04.061941 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e479994-13f1-41cf-9884-c9e767573754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:04.062564 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e479994-13f1-41cf-9884-c9e767573754 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:04.062895 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e479994-13f1-41cf-9884-c9e767573754 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:04.063191 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e479994-13f1-41cf-9884-c9e767573754 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:04.134931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e479994-13f1-41cf-9884-c9e767573754 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:04.136345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e479994-13f1-41cf-9884-c9e767573754 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:04.237889 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 450/892] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:24:03 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:24:04.238200 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:04.244714 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd3a05-b092-4ba3-804a-a6a425668fa4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:04.265354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:04.357716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd3a05-b092-4ba3-804a-a6a425668fa4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:04.399033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfe741-05fb-4758-93fe-5944266ac322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:04.399631 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfe741-05fb-4758-93fe-5944266ac322 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:04.399919 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfe741-05fb-4758-93fe-5944266ac322 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:04.400198 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfe741-05fb-4758-93fe-5944266ac322 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:04.411226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cfe741-05fb-4758-93fe-5944266ac322 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:04.412765 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cfe741-05fb-4758-93fe-5944266ac322 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:04.429773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b8cfe741-05fb-4758-93fe-5944266ac322 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:04.429773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:04.429773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:04.429773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:04.429773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:04.429773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:04.429773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:04.429773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:04.429773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:04.429773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:04.429773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:04.429773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:04.429773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:24:04.429773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:24:04.429773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:24:04.429773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:24:04.430777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:04.430777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:04.430777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:04.430777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:04.430777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:04.430777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:04.430777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:04.430777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:04.430777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:04.430777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:04.430777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:04.430777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:04.430777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:04.430777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:04.430777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:04.430777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:04.430777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:24:04.431719 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:24:04.431719 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:04.431719 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:04.431719 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 443/893] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:24:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:04.436506 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e479994-13f1-41cf-9884-c9e767573754 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:04.499868 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566698a3-04c1-4a26-861a-ffed044d18c8 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:04.500350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566698a3-04c1-4a26-861a-ffed044d18c8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:04.500527 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566698a3-04c1-4a26-861a-ffed044d18c8 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:04.502085 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 444/894] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:24:04.507502 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9b41b-6cbd-4c3c-89fb-51c9619edf29 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:04.507954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9b41b-6cbd-4c3c-89fb-51c9619edf29 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:04.508503 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9b41b-6cbd-4c3c-89fb-51c9619edf29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:04.529374 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e4a9b41b-6cbd-4c3c-89fb-51c9619edf29 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:04.530750 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e4a9b41b-6cbd-4c3c-89fb-51c9619edf29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:04.546677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e479994-13f1-41cf-9884-c9e767573754 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:04.588807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4defb3-8904-44de-8d2b-4faa7d87b031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:04.589425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4defb3-8904-44de-8d2b-4faa7d87b031 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:04.589706 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4defb3-8904-44de-8d2b-4faa7d87b031 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:04.589982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4defb3-8904-44de-8d2b-4faa7d87b031 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:04.590621 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4defb3-8904-44de-8d2b-4faa7d87b031 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:04.590932 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4defb3-8904-44de-8d2b-4faa7d87b031 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:04.592332 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4defb3-8904-44de-8d2b-4faa7d87b031 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:04.609507 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 451/895] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:04.610540 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4defb3-8904-44de-8d2b-4faa7d87b031 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:04.614101 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:04.631328 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 445/896] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:04 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:24:04.646557 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a9b41b-6cbd-4c3c-89fb-51c9619edf29 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:04.676358 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:04.730437 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4defb3-8904-44de-8d2b-4faa7d87b031 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:04.759973 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a9b41b-6cbd-4c3c-89fb-51c9619edf29 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:04.766945 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7312087-9f5b-46ad-9995-1583d06165bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:04.767880 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7312087-9f5b-46ad-9995-1583d06165bb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:04.768393 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7312087-9f5b-46ad-9995-1583d06165bb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:04.768875 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7312087-9f5b-46ad-9995-1583d06165bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:04.776417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7312087-9f5b-46ad-9995-1583d06165bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:04.778092 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7312087-9f5b-46ad-9995-1583d06165bb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:04.790261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-f7312087-9f5b-46ad-9995-1583d06165bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:24:04.790261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:04.790261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:04.790261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:04.790261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:04.790261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:04.790261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:04.790261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:04.790261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:04.790261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:04.790261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:04.790261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:04.790261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:24:04.790261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:24:04.790261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:24:04.791202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:24:04.791202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:04.791202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:04.791202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:04.791202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:04.791202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:04.791202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:04.791202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:04.791202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:04.791202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:04.791202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:04.791202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:04.791202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:04.791202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:04.791202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:04.791202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:04.791202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:04.792329 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:24:04.792329 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:24:04.792329 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:24:04.792329 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:24:04.792329 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:24:04.792329 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:24:04.792329 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:24:04.792329 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:04.794348 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 452/897] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:24:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:04.800131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7312087-9f5b-46ad-9995-1583d06165bb None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:04.802204 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921acf8-b417-47ef-ba78-50abeaaea8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:04.802898 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921acf8-b417-47ef-ba78-50abeaaea8c5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:04.803207 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921acf8-b417-47ef-ba78-50abeaaea8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:04.803579 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921acf8-b417-47ef-ba78-50abeaaea8c5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:04.804140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e921acf8-b417-47ef-ba78-50abeaaea8c5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:04.805655 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e921acf8-b417-47ef-ba78-50abeaaea8c5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:04.815029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-e921acf8-b417-47ef-ba78-50abeaaea8c5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:24:04.815029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:04.815029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:24:04.815029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:24:04.815029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:24:04.815029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:24:04.815029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:24:04.815029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:04.815029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:24:04.815029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:04.815029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:24:04.815029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:04.815029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:04.815029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:04.815029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:04.815029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:04.815029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:04.816249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:04.816249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:04.816249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:04.816249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:04.816249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:04.816249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:04.816249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:24:04.816249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:24:04.816249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:24:04.816249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:24:04.816249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:04.816249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:04.816249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:04.816249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:04.816249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:04.816249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:04.817243 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:04.817243 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:04.817243 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:04.817243 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:04.817243 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:04.817243 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:04.817243 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:04.817243 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:04.817243 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:04.817243 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:04.817243 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:24:04.817243 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:04.817243 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:24:04.817243 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:04.821346 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 446/898] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 10 12:24:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:04.822545 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921acf8-b417-47ef-ba78-50abeaaea8c5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:04.886210 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:04.910686 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7312087-9f5b-46ad-9995-1583d06165bb None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:04.924120 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:04.928170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921acf8-b417-47ef-ba78-50abeaaea8c5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:04.948427 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0829f46-ac47-439a-bd2a-ee9c791492ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:04.949039 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0829f46-ac47-439a-bd2a-ee9c791492ca None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:04.949746 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0829f46-ac47-439a-bd2a-ee9c791492ca None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:04.950033 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0829f46-ac47-439a-bd2a-ee9c791492ca None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:04.950721 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0829f46-ac47-439a-bd2a-ee9c791492ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:04.951031 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0829f46-ac47-439a-bd2a-ee9c791492ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:04.952491 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0829f46-ac47-439a-bd2a-ee9c791492ca None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:04.965013 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd87b855-7e78-404c-96ae-b92517ffeb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:04.965173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 453/899] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:24:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:04.965513 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87b855-7e78-404c-96ae-b92517ffeb1b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:04.968812 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87b855-7e78-404c-96ae-b92517ffeb1b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:04.968812 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87b855-7e78-404c-96ae-b92517ffeb1b None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:04.968812 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd87b855-7e78-404c-96ae-b92517ffeb1b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:04.968812 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd87b855-7e78-404c-96ae-b92517ffeb1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:04.968812 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd87b855-7e78-404c-96ae-b92517ffeb1b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:04.970912 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d376e-8d6e-47cc-b7a3-e79b717ad852 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:04.971211 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d376e-8d6e-47cc-b7a3-e79b717ad852 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:04.971475 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d376e-8d6e-47cc-b7a3-e79b717ad852 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:04.976763 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 454/900] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:04.976763 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d376e-8d6e-47cc-b7a3-e79b717ad852 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:04.978623 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 447/901] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 10 12:24:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:04.980146 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4d4b8-66d4-48bc-ad15-372248b3eb53 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:04.980443 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4d4b8-66d4-48bc-ad15-372248b3eb53 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:04.980880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4d4b8-66d4-48bc-ad15-372248b3eb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:04.999816 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-baf4d4b8-66d4-48bc-ad15-372248b3eb53 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:05.001047 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-baf4d4b8-66d4-48bc-ad15-372248b3eb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:05.068516 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 448/902] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:04 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:05.071080 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4d4b8-66d4-48bc-ad15-372248b3eb53 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:05.096374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d376e-8d6e-47cc-b7a3-e79b717ad852 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:05.133025 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da442c-930a-4565-8dce-2e04241923dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:05.133633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da442c-930a-4565-8dce-2e04241923dc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:05.133908 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da442c-930a-4565-8dce-2e04241923dc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:05.134182 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da442c-930a-4565-8dce-2e04241923dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:05.134898 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da442c-930a-4565-8dce-2e04241923dc None admin] RBAC: Adding query filter params (role.id=eb8debe34fab43f683a44c57759db243, scope.project.id=8e201e5f281a4977be503e22daf149d7, user.id=e42059e9af9842548ea08dcd39ff0b6e) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:05.135232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da442c-930a-4565-8dce-2e04241923dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:05.136636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da442c-930a-4565-8dce-2e04241923dc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:05.138333 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:05.142366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 455/903] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:24:04 2022] GET /identity/v3/role_assignments?user.id=e42059e9af9842548ea08dcd39ff0b6e&scope.project.id=8e201e5f281a4977be503e22daf149d7&role.id=eb8debe34fab43f683a44c57759db243 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:05.143140 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da442c-930a-4565-8dce-2e04241923dc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:05.195506 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4d4b8-66d4-48bc-ad15-372248b3eb53 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:05.196508 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:05.237059 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16e949-885d-48b9-9d9a-1948e48822b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:05.237935 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16e949-885d-48b9-9d9a-1948e48822b1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:05.238371 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16e949-885d-48b9-9d9a-1948e48822b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:05.238856 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16e949-885d-48b9-9d9a-1948e48822b1 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:05.239829 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16e949-885d-48b9-9d9a-1948e48822b1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:05.240136 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16e949-885d-48b9-9d9a-1948e48822b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:05.241647 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16e949-885d-48b9-9d9a-1948e48822b1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:05.247619 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 449/904] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 10 12:24:05 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:05.286288 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da442c-930a-4565-8dce-2e04241923dc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:05.330646 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf1076-f2b7-4e0d-a39e-e3e8d9085255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:05.331650 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf1076-f2b7-4e0d-a39e-e3e8d9085255 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:05.332184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf1076-f2b7-4e0d-a39e-e3e8d9085255 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:05.332659 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf1076-f2b7-4e0d-a39e-e3e8d9085255 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:05.333503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf1076-f2b7-4e0d-a39e-e3e8d9085255 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:05.335746 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf1076-f2b7-4e0d-a39e-e3e8d9085255 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:05.345865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-ecaf1076-f2b7-4e0d-a39e-e3e8d9085255 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 12:24:05.345865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:05.345865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:24:05.345865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:24:05.345865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:24:05.345865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:24:05.345865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:24:05.345865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:05.345865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 12:24:05.345865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:05.345865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:24:05.345865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:05.345865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:05.345865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:05.345865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:05.345865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:05.345865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:05.346921 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:05.346921 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:05.346921 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:05.346921 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:05.346921 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:05.346921 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:05.346921 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:24:05.346921 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:24:05.346921 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:24:05.346921 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:24:05.346921 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:05.346921 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:05.346921 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:05.346921 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:05.346921 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:05.346921 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:05.347798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:05.347798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:05.347798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:05.347798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:05.347798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:05.347798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:05.347798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:05.347798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:05.347798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:05.347798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:05.347798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:24:05.347798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:05.347798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 12:24:05.347798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:05.350366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 456/905] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 12:24:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:05.356814 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16e949-885d-48b9-9d9a-1948e48822b1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:05.468375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16e949-885d-48b9-9d9a-1948e48822b1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:05.487599 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:05.507147 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28979dcb-3f8a-4054-87c5-e9d2a9dcacd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:05.508099 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28979dcb-3f8a-4054-87c5-e9d2a9dcacd9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:05.508603 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28979dcb-3f8a-4054-87c5-e9d2a9dcacd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:05.509028 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28979dcb-3f8a-4054-87c5-e9d2a9dcacd9 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:05.509812 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28979dcb-3f8a-4054-87c5-e9d2a9dcacd9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:05.510266 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28979dcb-3f8a-4054-87c5-e9d2a9dcacd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:05.511803 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28979dcb-3f8a-4054-87c5-e9d2a9dcacd9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:05.517123 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 450/906] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 12:24:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:05.523109 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf1076-f2b7-4e0d-a39e-e3e8d9085255 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:05.629008 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:05.633217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf1076-f2b7-4e0d-a39e-e3e8d9085255 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:05.669662 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa11cb-0ea8-495f-9ae8-443d3d5f9a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:05.670287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa11cb-0ea8-495f-9ae8-443d3d5f9a31 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:05.670565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa11cb-0ea8-495f-9ae8-443d3d5f9a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:05.670838 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa11cb-0ea8-495f-9ae8-443d3d5f9a31 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:05.671495 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa11cb-0ea8-495f-9ae8-443d3d5f9a31 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:05.671801 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa11cb-0ea8-495f-9ae8-443d3d5f9a31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:05.673161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa11cb-0ea8-495f-9ae8-443d3d5f9a31 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:05.679016 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 457/907] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 12:24:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:05.684688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28979dcb-3f8a-4054-87c5-e9d2a9dcacd9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:05.795658 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28979dcb-3f8a-4054-87c5-e9d2a9dcacd9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:05.832283 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4456741b-6c4f-46f9-8f6d-c1cbacde7079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:05.833265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456741b-6c4f-46f9-8f6d-c1cbacde7079 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:05.833754 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456741b-6c4f-46f9-8f6d-c1cbacde7079 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:05.834230 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456741b-6c4f-46f9-8f6d-c1cbacde7079 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:05.835556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4456741b-6c4f-46f9-8f6d-c1cbacde7079 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:05.837145 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4456741b-6c4f-46f9-8f6d-c1cbacde7079 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:05.918041 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 451/908] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:24:05 2022] POST /identity/v3/endpoints => generated 346 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:24:06.013964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba873ca6-49c9-43ae-b7e8-bac4a715bbcd None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:06.014283 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba873ca6-49c9-43ae-b7e8-bac4a715bbcd None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:06.014523 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba873ca6-49c9-43ae-b7e8-bac4a715bbcd None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:06.016189 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 458/909] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:06.021702 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d74559-3ee9-4773-b177-95deaaeb8125 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:06.022401 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d74559-3ee9-4773-b177-95deaaeb8125 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:06.022401 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d74559-3ee9-4773-b177-95deaaeb8125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:06.044890 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c7d74559-3ee9-4773-b177-95deaaeb8125 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:06.046151 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c7d74559-3ee9-4773-b177-95deaaeb8125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:06.082789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:06.132351 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 452/910] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:06.155579 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba873ca6-49c9-43ae-b7e8-bac4a715bbcd None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:06.263652 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba873ca6-49c9-43ae-b7e8-bac4a715bbcd None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:06.300292 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265cf4e7-2f27-4d25-afb4-1ca5d937d867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:06.300857 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265cf4e7-2f27-4d25-afb4-1ca5d937d867 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:06.301139 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265cf4e7-2f27-4d25-afb4-1ca5d937d867 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:06.301416 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265cf4e7-2f27-4d25-afb4-1ca5d937d867 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:06.301998 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265cf4e7-2f27-4d25-afb4-1ca5d937d867 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:06.303926 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265cf4e7-2f27-4d25-afb4-1ca5d937d867 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:06.313520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-265cf4e7-2f27-4d25-afb4-1ca5d937d867 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 12:24:06.313520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:06.313520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:24:06.313520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:24:06.313520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:24:06.313520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:24:06.313520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:24:06.313520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:06.313520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 12:24:06.313520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:06.313520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:24:06.313520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:06.313520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:06.313520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:06.313520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:06.313520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:06.314532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:06.314532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:06.314532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:06.314532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:06.314532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:06.314532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:06.314532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:06.314532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:24:06.314532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:24:06.314532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:24:06.314532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:24:06.314532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:06.314532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:06.314532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:06.314532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:06.314532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:06.314532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:06.315481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:06.315481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:06.315481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:06.315481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:06.315481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:06.315481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:06.315481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:06.315481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:06.315481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:06.315481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:06.315481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:24:06.315481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:06.315481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 12:24:06.315481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:06.315481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 459/911] 10.222.0.49 () {56 vars in 1132 bytes} [Wed Aug 10 12:24:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:06.323913 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d74559-3ee9-4773-b177-95deaaeb8125 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:06.428057 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d74559-3ee9-4773-b177-95deaaeb8125 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:06.466743 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccc04c-435f-461d-8c1a-59c7dd7b8bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:06.467395 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccc04c-435f-461d-8c1a-59c7dd7b8bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:06.467683 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccc04c-435f-461d-8c1a-59c7dd7b8bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:06.467960 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccc04c-435f-461d-8c1a-59c7dd7b8bd8 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:06.468591 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dccc04c-435f-461d-8c1a-59c7dd7b8bd8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:06.468897 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dccc04c-435f-461d-8c1a-59c7dd7b8bd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:06.470273 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dccc04c-435f-461d-8c1a-59c7dd7b8bd8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:06.476060 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 453/912] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 10 12:24:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:06.484303 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265cf4e7-2f27-4d25-afb4-1ca5d937d867 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:06.514956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:06.591242 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265cf4e7-2f27-4d25-afb4-1ca5d937d867 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:06.627140 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f3107-26ec-4d7f-a159-b46f7b84e4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:06.627841 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f3107-26ec-4d7f-a159-b46f7b84e4cb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:06.628130 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f3107-26ec-4d7f-a159-b46f7b84e4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:06.628406 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f3107-26ec-4d7f-a159-b46f7b84e4cb None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:06.629032 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9f3107-26ec-4d7f-a159-b46f7b84e4cb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:06.629340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9f3107-26ec-4d7f-a159-b46f7b84e4cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:06.630690 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9f3107-26ec-4d7f-a159-b46f7b84e4cb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:06.636006 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 460/913] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 10 12:24:06 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:06.641678 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccc04c-435f-461d-8c1a-59c7dd7b8bd8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:06.679101 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:06.757788 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccc04c-435f-461d-8c1a-59c7dd7b8bd8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:06.792703 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2170b6c-d9a7-421a-ae3e-1b2b5e95d68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:06.793302 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2170b6c-d9a7-421a-ae3e-1b2b5e95d68f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:06.793578 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2170b6c-d9a7-421a-ae3e-1b2b5e95d68f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:06.793848 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2170b6c-d9a7-421a-ae3e-1b2b5e95d68f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:06.794535 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2170b6c-d9a7-421a-ae3e-1b2b5e95d68f None admin] RBAC: Adding query filter params (interface=public, service_id=e163da2b0cb549ec8aaba86dfe41e0bc, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:06.794841 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2170b6c-d9a7-421a-ae3e-1b2b5e95d68f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:06.796374 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2170b6c-d9a7-421a-ae3e-1b2b5e95d68f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:06.801722 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 454/914] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Aug 10 12:24:06 2022] GET /identity/v3/endpoints?service_id=e163da2b0cb549ec8aaba86dfe41e0bc&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:06.807649 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f3107-26ec-4d7f-a159-b46f7b84e4cb None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:06.903590 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:06.911968 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f3107-26ec-4d7f-a159-b46f7b84e4cb None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:06.951557 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf3515-366e-4840-8275-0fa24f6b648c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:06.952174 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf3515-366e-4840-8275-0fa24f6b648c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:06.952459 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf3515-366e-4840-8275-0fa24f6b648c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:06.952738 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf3515-366e-4840-8275-0fa24f6b648c None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:06.953345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf3515-366e-4840-8275-0fa24f6b648c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:06.954699 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf3515-366e-4840-8275-0fa24f6b648c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:06.961240 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 461/915] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Aug 10 12:24:06 2022] GET /identity/v3/services => generated 1268 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:24:07.308067 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387d9cb-029f-45a8-9484-dce5643d856e None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:07.308714 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387d9cb-029f-45a8-9484-dce5643d856e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:07.309190 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387d9cb-029f-45a8-9484-dce5643d856e None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:07.311030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 455/916] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:07.316259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbd0ea-5d26-4c00-982e-53f5e170c087 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:07.316783 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbd0ea-5d26-4c00-982e-53f5e170c087 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:07.317254 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbd0ea-5d26-4c00-982e-53f5e170c087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:07.337087 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e2dbd0ea-5d26-4c00-982e-53f5e170c087 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:07.338587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e2dbd0ea-5d26-4c00-982e-53f5e170c087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:07.410149 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 462/917] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:07 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:07.433391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0387d9cb-029f-45a8-9484-dce5643d856e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:07.516504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e083d1-0d53-433a-8d55-9c4b5ca34fac None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:07.517485 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e083d1-0d53-433a-8d55-9c4b5ca34fac None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:07.518018 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e083d1-0d53-433a-8d55-9c4b5ca34fac None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:07.520485 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 463/918] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:07.527307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a13526-c4d5-4217-b695-53ae928a369e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:07.527612 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a13526-c4d5-4217-b695-53ae928a369e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:07.527888 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a13526-c4d5-4217-b695-53ae928a369e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:07.543599 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:07.546752 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-18a13526-c4d5-4217-b695-53ae928a369e None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:07.548069 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-18a13526-c4d5-4217-b695-53ae928a369e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:07.551728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0387d9cb-029f-45a8-9484-dce5643d856e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:07.590343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad20d1-7212-4d9f-9037-42b3ebdba8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:07.591479 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad20d1-7212-4d9f-9037-42b3ebdba8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:07.591958 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad20d1-7212-4d9f-9037-42b3ebdba8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:07.592625 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad20d1-7212-4d9f-9037-42b3ebdba8c2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:07.604797 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aad20d1-7212-4d9f-9037-42b3ebdba8c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:24:07.610237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-6aad20d1-7212-4d9f-9037-42b3ebdba8c2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 12:24:07.610237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:07.610237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:07.610237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:07.610237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:07.610237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:07.610237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:07.610237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:07.610237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:07.610237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:07.610237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:07.610237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:07.610237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:24:07.610237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:24:07.610237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:24:07.610237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:24:07.611160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:24:07.611160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:24:07.611160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:24:07.611160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:24:07.611160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:07.611160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:07.611160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:07.611160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:07.611160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:07.611160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:07.611160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:07.611160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:07.611160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:07.611160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:07.611160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:07.611160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:07.612062 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:07.612062 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:07.612062 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:07.612062 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:07.612062 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:24:07.612062 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:24:07.612062 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:24:07.612062 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:24:07.612062 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:24:07.612062 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:24:07.612062 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 12:24:07.612062 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:07.619519 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 464/920] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:07 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:24:07.620033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 456/920] 10.222.0.49 () {56 vars in 1120 bytes} [Wed Aug 10 12:24:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:07.625371 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a13526-c4d5-4217-b695-53ae928a369e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:07.646091 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad20d1-7212-4d9f-9037-42b3ebdba8c2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:07.646948 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:07.734301 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a13526-c4d5-4217-b695-53ae928a369e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:07.763488 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad20d1-7212-4d9f-9037-42b3ebdba8c2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:07.774808 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adc9fa-0a2d-4792-a99d-383dc7df34f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:07.776040 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adc9fa-0a2d-4792-a99d-383dc7df34f4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:07.776464 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adc9fa-0a2d-4792-a99d-383dc7df34f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:07.777184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adc9fa-0a2d-4792-a99d-383dc7df34f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:07.778276 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14adc9fa-0a2d-4792-a99d-383dc7df34f4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:07.778752 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14adc9fa-0a2d-4792-a99d-383dc7df34f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:07.780800 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14adc9fa-0a2d-4792-a99d-383dc7df34f4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:07.792313 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 465/921] 10.222.0.49 () {56 vars in 1115 bytes} [Wed Aug 10 12:24:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:07.799018 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adc9fa-0a2d-4792-a99d-383dc7df34f4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:07.809549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b9a38-be45-4ecc-926f-69d5349ac171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:07.810167 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b9a38-be45-4ecc-926f-69d5349ac171 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:07.810451 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b9a38-be45-4ecc-926f-69d5349ac171 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:07.810726 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b9a38-be45-4ecc-926f-69d5349ac171 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:07.811432 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b9a38-be45-4ecc-926f-69d5349ac171 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:07.813270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b9a38-be45-4ecc-926f-69d5349ac171 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:07.887258 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:07.894563 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 457/922] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 10 12:24:07 2022] POST /identity/v3/services => generated 241 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:24:07.914250 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adc9fa-0a2d-4792-a99d-383dc7df34f4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:07.961865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cdc23e-878b-4ef0-a8d7-61f79f319aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:07.962584 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cdc23e-878b-4ef0-a8d7-61f79f319aff None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:07.962868 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cdc23e-878b-4ef0-a8d7-61f79f319aff None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:07.963142 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cdc23e-878b-4ef0-a8d7-61f79f319aff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:07.975982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cdc23e-878b-4ef0-a8d7-61f79f319aff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:07.977552 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cdc23e-878b-4ef0-a8d7-61f79f319aff None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:07.994447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-87cdc23e-878b-4ef0-a8d7-61f79f319aff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:07.994447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:07.994447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:07.994447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:07.994447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:07.994447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:07.994447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:07.994447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:07.994447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:07.994447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:07.994447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:07.994447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:07.994447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:24:07.994447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:24:07.994447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:24:07.994447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:24:07.995578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:07.995578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:07.995578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:07.995578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:07.995578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:07.995578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:07.995578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:07.995578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:07.995578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:07.995578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:07.995578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:07.995578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:07.995578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:07.995578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:07.995578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:07.995578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:07.995578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:24:07.997040 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:24:07.997040 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:07.997040 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:07.997040 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 466/923] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:24:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:08.003427 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b9a38-be45-4ecc-926f-69d5349ac171 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:08.016656 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b293ba81-83df-4214-998a-622172fe7b6c None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:08.016945 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b293ba81-83df-4214-998a-622172fe7b6c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:08.017331 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b293ba81-83df-4214-998a-622172fe7b6c None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:08.018858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 467/924] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:24:08.024046 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:08.025730 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c98ba-e34c-4312-a286-1159cb44ab3a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:08.026251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c98ba-e34c-4312-a286-1159cb44ab3a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:08.026813 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c98ba-e34c-4312-a286-1159cb44ab3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:08.050374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-8e1c98ba-e34c-4312-a286-1159cb44ab3a None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:08.051816 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8e1c98ba-e34c-4312-a286-1159cb44ab3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:08.126551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b9a38-be45-4ecc-926f-69d5349ac171 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:08.138548 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 468/925] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:24:08.162388 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c98ba-e34c-4312-a286-1159cb44ab3a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:08.164619 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f6815-13f3-4b8b-943b-a3f418ed2bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:08.165285 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f6815-13f3-4b8b-943b-a3f418ed2bcd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:08.165556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f6815-13f3-4b8b-943b-a3f418ed2bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:08.165829 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f6815-13f3-4b8b-943b-a3f418ed2bcd None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:08.166499 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f6815-13f3-4b8b-943b-a3f418ed2bcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:08.166805 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f6815-13f3-4b8b-943b-a3f418ed2bcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:08.168265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f6815-13f3-4b8b-943b-a3f418ed2bcd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:08.185098 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 458/926] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:08.200899 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f6815-13f3-4b8b-943b-a3f418ed2bcd None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:08.235052 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:08.292861 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c98ba-e34c-4312-a286-1159cb44ab3a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:08.295399 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:08.327143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f6815-13f3-4b8b-943b-a3f418ed2bcd None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:08.332838 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d31edf-9e23-48af-a632-cf9d1fa2aa0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:08.333620 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d31edf-9e23-48af-a632-cf9d1fa2aa0c None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:08.333909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d31edf-9e23-48af-a632-cf9d1fa2aa0c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:08.334183 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d31edf-9e23-48af-a632-cf9d1fa2aa0c None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:08.334891 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d31edf-9e23-48af-a632-cf9d1fa2aa0c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:08.337350 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d31edf-9e23-48af-a632-cf9d1fa2aa0c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:08.340301 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: WARNING keystone.server.flask.common [None req-d5d31edf-9e23-48af-a632-cf9d1fa2aa0c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 10 12:24:08.365110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0626c21-3bc0-4132-b76c-db6bfa7faa71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:08.365765 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0626c21-3bc0-4132-b76c-db6bfa7faa71 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:08.366764 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0626c21-3bc0-4132-b76c-db6bfa7faa71 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:08.367039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0626c21-3bc0-4132-b76c-db6bfa7faa71 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:08.375314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0626c21-3bc0-4132-b76c-db6bfa7faa71 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:08.376752 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0626c21-3bc0-4132-b76c-db6bfa7faa71 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:08.388037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f0626c21-3bc0-4132-b76c-db6bfa7faa71 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 12:24:08.388037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:08.388037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:08.388037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:08.388037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:08.388037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:08.388037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:08.388037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:08.388037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:08.388037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:08.388037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:08.388037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:08.388037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:24:08.388037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:24:08.388037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:24:08.388037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:24:08.389071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:08.389071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:08.389071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:24:08.389071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:24:08.389071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:24:08.389071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:24:08.389071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:08.389071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:08.389071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:08.389071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:08.389071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:08.389071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:08.389071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:08.389071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:08.389071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:08.389071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:08.389071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:08.389071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:08.394905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:08.394905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:08.394905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:24:08.394905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:24:08.394905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:24:08.394905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:24:08.394905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:24:08.394905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:24:08.394905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 12:24:08.394905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:08.394905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 459/927] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 10 12:24:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:08.396961 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0626c21-3bc0-4132-b76c-db6bfa7faa71 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:08.398201 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.sql.core [None req-d5d31edf-9e23-48af-a632-cf9d1fa2aa0c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 12:24:08.398201 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 10 12:24:08.398201 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [parameters: {'id': '43dea538dc43457c853a5dab84d6dcc5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 10 12:24:08.398201 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80202) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 12:24:08.408825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-d5d31edf-9e23-48af-a632-cf9d1fa2aa0c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 43dea538dc43457c853a5dab84d6dcc5.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 43dea538dc43457c853a5dab84d6dcc5. Aug 10 12:24:08.408825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:08.408825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 12:24:08.408825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 12:24:08.408825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 12:24:08.408825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 12:24:08.408825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 12:24:08.408825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 12:24:08.408825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 12:24:08.408825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application conn.query(q) Aug 10 12:24:08.408825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 12:24:08.408825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 12:24:08.408825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 12:24:08.409673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application result.read() Aug 10 12:24:08.409673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 12:24:08.409673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 12:24:08.409673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 12:24:08.409673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 12:24:08.409673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 12:24:08.409673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 12:24:08.409673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 12:24:08.409673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 12:24:08.409673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 12:24:08.409673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:08.409673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 12:24:08.409673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:08.409673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:08.409673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 12:24:08.409673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 12:24:08.409673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 10 12:24:08.409673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 10 12:24:08.410653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 12:24:08.410653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application next(self.gen) Aug 10 12:24:08.410653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 12:24:08.410653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application yield resource Aug 10 12:24:08.410653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 12:24:08.410653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application next(self.gen) Aug 10 12:24:08.410653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 12:24:08.410653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 12:24:08.410653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 12:24:08.410653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 12:24:08.410653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 12:24:08.410653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise self.value Aug 10 12:24:08.410653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 12:24:08.410653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 12:24:08.410653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 12:24:08.410653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application session.commit() Aug 10 12:24:08.410653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 12:24:08.411616 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 12:24:08.411616 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 12:24:08.411616 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 12:24:08.411616 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 12:24:08.411616 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self.session.flush() Aug 10 12:24:08.411616 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 12:24:08.411616 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 12:24:08.411616 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 12:24:08.411616 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 12:24:08.411616 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 12:24:08.411616 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application compat.raise_( Aug 10 12:24:08.411616 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 12:24:08.411616 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception Aug 10 12:24:08.411616 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 12:24:08.411616 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 12:24:08.411616 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 12:24:08.411616 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 12:24:08.412574 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 12:24:08.412574 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 12:24:08.412574 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 12:24:08.412574 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 12:24:08.412574 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 10 12:24:08.412574 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 10 12:24:08.412574 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 12:24:08.412574 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 12:24:08.412574 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 12:24:08.412574 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 12:24:08.412574 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 12:24:08.412574 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 12:24:08.412574 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 12:24:08.412574 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 12:24:08.412574 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 12:24:08.413930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 12:24:08.413930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 12:24:08.413930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception Aug 10 12:24:08.413930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 12:24:08.413930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 12:24:08.413930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 12:24:08.413930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 12:24:08.413930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 12:24:08.413930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 12:24:08.413930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 12:24:08.413930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application conn.query(q) Aug 10 12:24:08.413930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 12:24:08.413930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 12:24:08.413930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 12:24:08.413930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application result.read() Aug 10 12:24:08.413930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 12:24:08.413930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 12:24:08.414833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 12:24:08.414833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 12:24:08.414833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 12:24:08.414833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 12:24:08.414833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 12:24:08.414833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 12:24:08.414833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 12:24:08.414833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 10 12:24:08.414833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [parameters: {'id': '43dea538dc43457c853a5dab84d6dcc5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 10 12:24:08.414833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 12:24:08.414833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:08.414833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:24:08.414833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:08.414833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:08.414833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 10 12:24:08.417572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 10 12:24:08.417572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 12:24:08.417572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 12:24:08.417572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 10 12:24:08.417572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:08.417572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:24:08.417572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:08.417572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:08.417572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:08.417572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:08.417572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:08.417572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:08.417572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:08.417572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:08.417572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:08.417572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:08.417572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:08.420030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:08.420030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 10 12:24:08.420030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 10 12:24:08.420030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:08.420030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:08.420030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 10 12:24:08.420030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 10 12:24:08.420030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 43dea538dc43457c853a5dab84d6dcc5. Aug 10 12:24:08.420030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:08.420030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 469/928] 10.222.0.49 () {60 vars in 1140 bytes} [Wed Aug 10 12:24:08 2022] POST /identity/v3/projects => generated 262 bytes in 257 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:24:08.424761 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d31edf-9e23-48af-a632-cf9d1fa2aa0c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:08.535663 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d31edf-9e23-48af-a632-cf9d1fa2aa0c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:08.535874 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0626c21-3bc0-4132-b76c-db6bfa7faa71 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:08.558848 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:08.568280 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:08.575193 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da10a78-a00e-4589-b20f-5fd4dccc9982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:08.576033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da10a78-a00e-4589-b20f-5fd4dccc9982 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:08.576497 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da10a78-a00e-4589-b20f-5fd4dccc9982 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:08.576953 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da10a78-a00e-4589-b20f-5fd4dccc9982 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:08.577917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da10a78-a00e-4589-b20f-5fd4dccc9982 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:08.578425 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da10a78-a00e-4589-b20f-5fd4dccc9982 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:08.580062 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da10a78-a00e-4589-b20f-5fd4dccc9982 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:08.585853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea6cc8-a317-4386-b1a6-168070f086a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:08.586493 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea6cc8-a317-4386-b1a6-168070f086a2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:08.586774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea6cc8-a317-4386-b1a6-168070f086a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:08.587048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea6cc8-a317-4386-b1a6-168070f086a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:08.596752 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea6cc8-a317-4386-b1a6-168070f086a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:08.598421 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea6cc8-a317-4386-b1a6-168070f086a2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:08.606906 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 460/929] 10.222.0.49 () {56 vars in 1149 bytes} [Wed Aug 10 12:24:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:08.608976 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da10a78-a00e-4589-b20f-5fd4dccc9982 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:08.610893 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-c2ea6cc8-a317-4386-b1a6-168070f086a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:24:08.610893 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:08.610893 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:08.610893 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:08.610893 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:08.610893 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:08.610893 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:08.610893 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:08.610893 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:08.610893 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:08.610893 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:08.610893 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:08.610893 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:24:08.610893 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:24:08.610893 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:24:08.611835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:24:08.611835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:08.611835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:08.611835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:08.611835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:08.611835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:08.611835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:08.611835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:08.611835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:08.611835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:08.611835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:08.611835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:08.611835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:08.611835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:08.611835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:08.611835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:08.611835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:08.613065 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:24:08.613065 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:24:08.613065 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:24:08.613065 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:24:08.613065 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:24:08.613065 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:24:08.613065 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:24:08.613065 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:08.615254 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 470/930] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:24:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:08.619142 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea6cc8-a317-4386-b1a6-168070f086a2 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:08.744128 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da10a78-a00e-4589-b20f-5fd4dccc9982 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:08.745062 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea6cc8-a317-4386-b1a6-168070f086a2 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:08.781026 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f1109-3de9-4e24-be44-3d54fdd40e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:08.781654 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f1109-3de9-4e24-be44-3d54fdd40e88 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:08.782020 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f1109-3de9-4e24-be44-3d54fdd40e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:08.782358 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f1109-3de9-4e24-be44-3d54fdd40e88 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:08.783253 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4f1109-3de9-4e24-be44-3d54fdd40e88 None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:08.783667 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4f1109-3de9-4e24-be44-3d54fdd40e88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:08.784904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8acd94-4925-40bb-8d26-461066a42ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:08.785370 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4f1109-3de9-4e24-be44-3d54fdd40e88 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:08.786126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8acd94-4925-40bb-8d26-461066a42ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:08.786649 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8acd94-4925-40bb-8d26-461066a42ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:08.787062 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8acd94-4925-40bb-8d26-461066a42ff0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:08.798205 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8acd94-4925-40bb-8d26-461066a42ff0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:08.800005 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8acd94-4925-40bb-8d26-461066a42ff0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:08.815328 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 471/931] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 10 12:24:08 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:08.824863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-fc8acd94-4925-40bb-8d26-461066a42ff0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:08.824863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:08.824863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:08.824863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:08.824863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:08.824863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:08.824863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:08.824863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:08.824863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:08.824863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:08.824863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:08.824863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:08.824863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:24:08.824863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:24:08.824863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:24:08.824863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:24:08.826109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:08.826109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:08.826109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:08.826109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:08.826109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:08.826109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:08.826109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:08.826109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:08.826109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:08.826109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:08.826109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:08.826109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:08.826109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:08.826109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:08.826109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:08.826109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:08.826109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:24:08.827227 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:24:08.827227 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:08.827227 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:08.833617 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 461/932] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:24:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:08.835689 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f1109-3de9-4e24-be44-3d54fdd40e88 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:08.872792 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:08.951011 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f1109-3de9-4e24-be44-3d54fdd40e88 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:08.991076 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0fbfb6-8367-4ce7-9856-a85efd33b13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:08.991931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fbfb6-8367-4ce7-9856-a85efd33b13f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:08.992209 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fbfb6-8367-4ce7-9856-a85efd33b13f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:08.992480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fbfb6-8367-4ce7-9856-a85efd33b13f None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:08.993124 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0fbfb6-8367-4ce7-9856-a85efd33b13f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:08.993431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0fbfb6-8367-4ce7-9856-a85efd33b13f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:08.994791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0fbfb6-8367-4ce7-9856-a85efd33b13f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:09.008047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 472/933] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:09.013635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8acd94-4925-40bb-8d26-461066a42ff0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:09.060493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:09.092738 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d7f1e-1b13-4a8b-b03c-3ce2a6f83cf0 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:09.092936 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d7f1e-1b13-4a8b-b03c-3ce2a6f83cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:09.093218 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d7f1e-1b13-4a8b-b03c-3ce2a6f83cf0 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:09.095602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 473/934] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:24:09.102227 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb1695-4805-4d6d-9d5b-e3f96ab43cc2 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:09.102532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb1695-4805-4d6d-9d5b-e3f96ab43cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:09.102775 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb1695-4805-4d6d-9d5b-e3f96ab43cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:09.126148 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8acd94-4925-40bb-8d26-461066a42ff0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:09.128227 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-23cb1695-4805-4d6d-9d5b-e3f96ab43cc2 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:09.130828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-23cb1695-4805-4d6d-9d5b-e3f96ab43cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:09.169052 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d80f0-42d5-4291-80bc-534399d5a97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:09.170226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d80f0-42d5-4291-80bc-534399d5a97b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:09.171004 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d80f0-42d5-4291-80bc-534399d5a97b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:09.171760 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d80f0-42d5-4291-80bc-534399d5a97b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:09.180446 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d80f0-42d5-4291-80bc-534399d5a97b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:09.182626 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d80f0-42d5-4291-80bc-534399d5a97b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:09.198077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-950d80f0-42d5-4291-80bc-534399d5a97b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:24:09.198077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:09.198077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:09.198077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:09.198077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:09.198077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:09.198077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:09.198077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:09.198077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:09.198077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:09.198077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:09.198077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:09.198077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:24:09.198077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:24:09.198077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:24:09.199159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:24:09.199159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:09.199159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:09.199159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:09.199159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:09.199159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:09.199159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:09.199159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:09.199159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:09.199159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:09.199159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:09.199159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:09.199159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:09.199159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:09.199159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:09.199159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:09.199159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:09.200094 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:24:09.200094 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:24:09.200094 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:24:09.200094 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:24:09.200094 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:24:09.200094 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:24:09.200094 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:24:09.200094 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:09.201820 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 462/935] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:24:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:09.205984 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d80f0-42d5-4291-80bc-534399d5a97b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:09.217371 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 474/936] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:09.239423 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb1695-4805-4d6d-9d5b-e3f96ab43cc2 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:09.270620 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:09.329976 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d80f0-42d5-4291-80bc-534399d5a97b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:09.332188 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:09.360807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb1695-4805-4d6d-9d5b-e3f96ab43cc2 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:09.377561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32099ffe-691b-4db1-b4c9-a211f8df80eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:09.378618 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32099ffe-691b-4db1-b4c9-a211f8df80eb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:09.378897 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32099ffe-691b-4db1-b4c9-a211f8df80eb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:09.379168 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32099ffe-691b-4db1-b4c9-a211f8df80eb None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:09.380177 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32099ffe-691b-4db1-b4c9-a211f8df80eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:09.380485 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32099ffe-691b-4db1-b4c9-a211f8df80eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:09.382245 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32099ffe-691b-4db1-b4c9-a211f8df80eb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:09.403261 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 463/937] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:24:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:09.404148 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9f1af-6896-477f-8a12-75a19e09294d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:09.405030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9f1af-6896-477f-8a12-75a19e09294d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:09.406301 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32099ffe-691b-4db1-b4c9-a211f8df80eb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:09.406780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9f1af-6896-477f-8a12-75a19e09294d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:09.407063 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9f1af-6896-477f-8a12-75a19e09294d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:09.407672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9f1af-6896-477f-8a12-75a19e09294d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:09.409032 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9f1af-6896-477f-8a12-75a19e09294d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:09.419717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-edd9f1af-6896-477f-8a12-75a19e09294d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 12:24:09.419717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:09.419717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:24:09.419717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:24:09.419717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:24:09.419717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:24:09.419717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:24:09.419717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:09.419717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 12:24:09.419717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:09.419717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:24:09.419717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:09.419717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:09.419717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:09.419717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:09.419717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:09.420705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:09.420705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:09.420705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:09.420705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:09.420705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:09.420705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:09.420705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:09.420705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:24:09.420705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:24:09.420705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:24:09.420705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:24:09.420705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:09.420705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:09.420705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:09.420705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:09.420705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:09.420705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:09.422230 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:09.422230 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:09.422230 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:09.422230 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:09.422230 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:09.422230 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:09.422230 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:09.422230 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:09.422230 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:09.422230 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:09.422230 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:24:09.422230 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:09.422230 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 12:24:09.422230 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:09.425929 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 475/938] 10.222.0.49 () {56 vars in 1132 bytes} [Wed Aug 10 12:24:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:09.426889 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9f1af-6896-477f-8a12-75a19e09294d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:09.533618 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:09.544716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32099ffe-691b-4db1-b4c9-a211f8df80eb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:09.552262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9f1af-6896-477f-8a12-75a19e09294d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:09.581627 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68240e6-b0be-453e-a655-6cb049e74319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:09.582686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68240e6-b0be-453e-a655-6cb049e74319 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:09.583211 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68240e6-b0be-453e-a655-6cb049e74319 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:09.583790 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68240e6-b0be-453e-a655-6cb049e74319 None admin] PATH_INFO: `/v3/projects/8e201e5f281a4977be503e22daf149d7/users/e42059e9af9842548ea08dcd39ff0b6e/roles/eb8debe34fab43f683a44c57759db243` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:09.587000 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68240e6-b0be-453e-a655-6cb049e74319 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e201e5f281a4977be503e22daf149d7, user_id=e42059e9af9842548ea08dcd39ff0b6e, role_id=eb8debe34fab43f683a44c57759db243)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:09.589152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68240e6-b0be-453e-a655-6cb049e74319 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:09.595162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa157b44-aba2-4604-82d6-597389182af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:09.595859 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa157b44-aba2-4604-82d6-597389182af4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:09.596140 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa157b44-aba2-4604-82d6-597389182af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:09.596413 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa157b44-aba2-4604-82d6-597389182af4 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:09.597094 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa157b44-aba2-4604-82d6-597389182af4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:09.597403 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa157b44-aba2-4604-82d6-597389182af4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:09.598809 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa157b44-aba2-4604-82d6-597389182af4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:09.609291 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 476/939] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 10 12:24:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:09.610788 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa157b44-aba2-4604-82d6-597389182af4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:09.710339 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 464/940] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 12:24:09 2022] PUT /identity/v3/projects/8e201e5f281a4977be503e22daf149d7/users/e42059e9af9842548ea08dcd39ff0b6e/roles/eb8debe34fab43f683a44c57759db243 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:24:09.716462 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa157b44-aba2-4604-82d6-597389182af4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:09.755879 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3e046-84b2-4fec-b719-992f3f3bfe07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:09.757043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3e046-84b2-4fec-b719-992f3f3bfe07 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:09.757347 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3e046-84b2-4fec-b719-992f3f3bfe07 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:09.757834 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3e046-84b2-4fec-b719-992f3f3bfe07 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:09.758510 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3e046-84b2-4fec-b719-992f3f3bfe07 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:09.758820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3e046-84b2-4fec-b719-992f3f3bfe07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:09.760324 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3e046-84b2-4fec-b719-992f3f3bfe07 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:09.766464 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 477/941] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 10 12:24:09 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:09.773502 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68240e6-b0be-453e-a655-6cb049e74319 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:09.807981 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:09.892524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68240e6-b0be-453e-a655-6cb049e74319 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:09.930045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed876ba-123e-4fdd-ab35-0bbecbf2bf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:09.930663 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed876ba-123e-4fdd-ab35-0bbecbf2bf0a None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:09.930943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed876ba-123e-4fdd-ab35-0bbecbf2bf0a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:09.931293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed876ba-123e-4fdd-ab35-0bbecbf2bf0a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:09.932155 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed876ba-123e-4fdd-ab35-0bbecbf2bf0a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:09.933528 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed876ba-123e-4fdd-ab35-0bbecbf2bf0a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:10.003996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 465/942] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:24:09 2022] POST /identity/v3/endpoints => generated 347 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:24:10.015915 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244b09c-32cb-4d89-bcf0-179181259d54 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:10.016255 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244b09c-32cb-4d89-bcf0-179181259d54 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:10.016467 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244b09c-32cb-4d89-bcf0-179181259d54 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:10.017897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 478/943] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:24:10.023987 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e575c-a258-48e1-8ef7-182bd48d4c49 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:10.026210 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e575c-a258-48e1-8ef7-182bd48d4c49 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:10.026210 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e575c-a258-48e1-8ef7-182bd48d4c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:10.046798 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7c5e575c-a258-48e1-8ef7-182bd48d4c49 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:10.048156 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7c5e575c-a258-48e1-8ef7-182bd48d4c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:10.137609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 466/944] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:10.160066 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3244b09c-32cb-4d89-bcf0-179181259d54 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:10.178926 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:10.274235 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3244b09c-32cb-4d89-bcf0-179181259d54 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:10.313196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af50453-828f-4b08-9c54-aa06be797e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:10.314117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af50453-828f-4b08-9c54-aa06be797e34 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:10.314607 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af50453-828f-4b08-9c54-aa06be797e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:10.315031 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af50453-828f-4b08-9c54-aa06be797e34 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:10.315835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af50453-828f-4b08-9c54-aa06be797e34 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:10.317418 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af50453-828f-4b08-9c54-aa06be797e34 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:10.327492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-1af50453-828f-4b08-9c54-aa06be797e34 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:24:10.327492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:10.327492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:24:10.327492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:24:10.327492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:24:10.327492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:24:10.327492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:24:10.327492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:10.327492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:24:10.327492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:10.327492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:24:10.327492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:10.327492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:10.327492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:10.327492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:10.327492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:10.327492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:10.328810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:10.328810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:10.328810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:10.328810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:10.328810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:10.328810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:10.328810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:24:10.328810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:24:10.328810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:24:10.328810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:24:10.328810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:10.328810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:10.328810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:10.328810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:10.328810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:10.328810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:10.329960 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:10.329960 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:10.329960 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:10.329960 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:10.329960 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:10.329960 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:10.329960 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:10.329960 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:10.329960 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:10.329960 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:10.329960 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:24:10.329960 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:10.329960 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:24:10.329960 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:10.332771 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 479/945] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 10 12:24:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:10.340482 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e575c-a258-48e1-8ef7-182bd48d4c49 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:10.341804 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:10.442197 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e575c-a258-48e1-8ef7-182bd48d4c49 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:10.482632 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0b104-fc09-49db-b86d-1f891173afb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:10.483625 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0b104-fc09-49db-b86d-1f891173afb3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:10.484171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0b104-fc09-49db-b86d-1f891173afb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:10.484669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0b104-fc09-49db-b86d-1f891173afb3 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:10.485608 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0b104-fc09-49db-b86d-1f891173afb3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:10.486161 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0b104-fc09-49db-b86d-1f891173afb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:10.487781 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0b104-fc09-49db-b86d-1f891173afb3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:10.494453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 467/946] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 10 12:24:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:10.500112 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af50453-828f-4b08-9c54-aa06be797e34 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:10.582930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:10.608668 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af50453-828f-4b08-9c54-aa06be797e34 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:10.646809 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a350b-fcca-4940-89f9-65961e226b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:10.647486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a350b-fcca-4940-89f9-65961e226b9c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:10.647764 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a350b-fcca-4940-89f9-65961e226b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:10.648038 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a350b-fcca-4940-89f9-65961e226b9c None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:10.648663 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a350b-fcca-4940-89f9-65961e226b9c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:10.649023 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a350b-fcca-4940-89f9-65961e226b9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:10.650437 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a350b-fcca-4940-89f9-65961e226b9c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:10.655698 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 480/947] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 10 12:24:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:10.661568 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0b104-fc09-49db-b86d-1f891173afb3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:10.732527 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:10.769452 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0b104-fc09-49db-b86d-1f891173afb3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:10.805864 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f98f14-068e-436d-885f-06ba8c41b5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:10.806869 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f98f14-068e-436d-885f-06ba8c41b5b6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:10.807474 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f98f14-068e-436d-885f-06ba8c41b5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:10.807967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f98f14-068e-436d-885f-06ba8c41b5b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:10.808896 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f98f14-068e-436d-885f-06ba8c41b5b6 None admin] RBAC: Adding query filter params (interface=public, service_id=8c82165e46d34ab5b63f4d2741cd059e, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:10.809841 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f98f14-068e-436d-885f-06ba8c41b5b6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:10.811447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f98f14-068e-436d-885f-06ba8c41b5b6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:10.817587 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 468/948] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Aug 10 12:24:10 2022] GET /identity/v3/endpoints?service_id=8c82165e46d34ab5b63f4d2741cd059e&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:10.827270 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a350b-fcca-4940-89f9-65961e226b9c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:10.932565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a350b-fcca-4940-89f9-65961e226b9c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:10.970400 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115e1a3-754c-4c96-9def-9c251f4265ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:10.973917 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115e1a3-754c-4c96-9def-9c251f4265ca None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:10.973917 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115e1a3-754c-4c96-9def-9c251f4265ca None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:10.973917 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115e1a3-754c-4c96-9def-9c251f4265ca None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:10.973917 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115e1a3-754c-4c96-9def-9c251f4265ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:10.973917 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115e1a3-754c-4c96-9def-9c251f4265ca None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:10.984270 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 481/949] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Aug 10 12:24:10 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:24:11.032765 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff480fe-58bf-4a31-8c6f-a15183666b1c None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:11.033062 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff480fe-58bf-4a31-8c6f-a15183666b1c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:11.033329 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff480fe-58bf-4a31-8c6f-a15183666b1c None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:11.034810 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 469/950] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:24:11.040822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d2872-0607-4644-895b-9637b74c203e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:11.041448 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d2872-0607-4644-895b-9637b74c203e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:11.042040 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d2872-0607-4644-895b-9637b74c203e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:11.057559 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:11.063403 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3c7d2872-0607-4644-895b-9637b74c203e None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:11.064613 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3c7d2872-0607-4644-895b-9637b74c203e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:11.138799 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 482/951] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:11.164237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff480fe-58bf-4a31-8c6f-a15183666b1c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:11.275283 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff480fe-58bf-4a31-8c6f-a15183666b1c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:11.313695 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985978c9-3d7e-472b-b77f-fa143d52c5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:11.314695 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985978c9-3d7e-472b-b77f-fa143d52c5ab None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:11.315201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985978c9-3d7e-472b-b77f-fa143d52c5ab None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:11.315708 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985978c9-3d7e-472b-b77f-fa143d52c5ab None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:11.316543 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985978c9-3d7e-472b-b77f-fa143d52c5ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:11.318168 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985978c9-3d7e-472b-b77f-fa143d52c5ab None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:11.321670 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: WARNING keystone.server.flask.common [None req-985978c9-3d7e-472b-b77f-fa143d52c5ab None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 10 12:24:11.324840 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:11.389581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.sql.core [None req-985978c9-3d7e-472b-b77f-fa143d52c5ab None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 12:24:11.389581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 10 12:24:11.389581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [parameters: {'user_id': 'd19a0a05c6614d72b472d8125de11155', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 10 12:24:11.389581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80203) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 12:24:11.429581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-985978c9-3d7e-472b-b77f-fa143d52c5ab None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 10 12:24:11.429581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:11.429581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 12:24:11.429581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 12:24:11.429581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 12:24:11.429581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 12:24:11.429581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 12:24:11.429581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 12:24:11.429581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 12:24:11.429581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application conn.query(q) Aug 10 12:24:11.429581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 12:24:11.429581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 12:24:11.429581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 12:24:11.429581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result.read() Aug 10 12:24:11.429581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 12:24:11.430601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 12:24:11.430601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 12:24:11.430601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 12:24:11.430601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 12:24:11.430601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 12:24:11.430601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 12:24:11.430601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 12:24:11.430601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 12:24:11.430601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:11.430601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 12:24:11.430601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:11.430601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:11.430601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 12:24:11.430601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 12:24:11.430601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 10 12:24:11.430601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 10 12:24:11.430601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 12:24:11.430601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application next(self.gen) Aug 10 12:24:11.431723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 12:24:11.431723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application yield resource Aug 10 12:24:11.431723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 12:24:11.431723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application next(self.gen) Aug 10 12:24:11.431723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 12:24:11.431723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 12:24:11.431723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 12:24:11.431723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 12:24:11.431723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 12:24:11.431723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise self.value Aug 10 12:24:11.431723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 12:24:11.431723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 12:24:11.431723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 12:24:11.431723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application session.commit() Aug 10 12:24:11.431723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 12:24:11.431723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 12:24:11.431723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 12:24:11.432640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 12:24:11.432640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 12:24:11.432640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.session.flush() Aug 10 12:24:11.432640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 12:24:11.432640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 12:24:11.432640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 12:24:11.432640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 12:24:11.432640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 12:24:11.432640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application compat.raise_( Aug 10 12:24:11.432640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 12:24:11.432640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception Aug 10 12:24:11.432640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 12:24:11.432640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 12:24:11.432640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 12:24:11.432640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 12:24:11.432640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 12:24:11.432640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 12:24:11.433561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 12:24:11.433561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 12:24:11.433561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 10 12:24:11.433561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 10 12:24:11.433561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 12:24:11.433561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 12:24:11.433561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 12:24:11.433561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 12:24:11.433561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 12:24:11.433561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 12:24:11.433561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 12:24:11.433561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 12:24:11.433561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 12:24:11.433561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 12:24:11.433561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 12:24:11.433561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception Aug 10 12:24:11.434532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 12:24:11.434532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 12:24:11.434532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 12:24:11.434532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 12:24:11.434532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 12:24:11.434532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 12:24:11.434532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 12:24:11.434532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application conn.query(q) Aug 10 12:24:11.434532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 12:24:11.434532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 12:24:11.434532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 12:24:11.434532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result.read() Aug 10 12:24:11.434532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 12:24:11.434532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 12:24:11.434532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 12:24:11.434532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 12:24:11.436769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 12:24:11.436769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 12:24:11.436769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 12:24:11.436769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 12:24:11.436769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 12:24:11.436769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 10 12:24:11.436769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd19a0a05c6614d72b472d8125de11155', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 10 12:24:11.436769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 12:24:11.436769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:11.436769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:24:11.436769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:11.436769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:11.436769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:11.436769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:11.436769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:11.438644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:11.438644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:11.438644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:11.438644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:11.438644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:11.438644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:11.438644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:11.438644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 10 12:24:11.438644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 10 12:24:11.438644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:11.438644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:11.438644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:24:11.438644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:24:11.438644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:24:11.438644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:24:11.438644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 10 12:24:11.438644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 10 12:24:11.448220 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 10 12:24:11.448220 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 10 12:24:11.448220 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 12:24:11.448220 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 12:24:11.448220 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 10 12:24:11.448220 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:11.448220 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 470/952] 10.222.0.49 () {60 vars in 1131 bytes} [Wed Aug 10 12:24:11 2022] POST /identity/v3/users => generated 160 bytes in 267 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:11.449255 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d2872-0607-4644-895b-9637b74c203e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:11.605645 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d2872-0607-4644-895b-9637b74c203e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:11.671108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e42078-8339-4fe6-b6ce-13003803d6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:11.671782 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e42078-8339-4fe6-b6ce-13003803d6ee None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:11.672059 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e42078-8339-4fe6-b6ce-13003803d6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:11.675196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e42078-8339-4fe6-b6ce-13003803d6ee None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:11.680215 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:11.690736 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e42078-8339-4fe6-b6ce-13003803d6ee None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:11.697117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e42078-8339-4fe6-b6ce-13003803d6ee None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:11.719232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-f6e42078-8339-4fe6-b6ce-13003803d6ee None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 12:24:11.719232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:11.719232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:11.719232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:11.719232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:11.719232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:11.719232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:11.719232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:11.719232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:11.719232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:11.719232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:11.719232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:11.719232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:24:11.719232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:24:11.719232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:24:11.719232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:24:11.720259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:11.720259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:11.720259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:24:11.720259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:24:11.720259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:24:11.720259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:24:11.720259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:11.720259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:11.720259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:11.720259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:11.720259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:11.720259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:11.720259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:11.720259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:11.720259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:11.720259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:11.720259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:11.720259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:11.721273 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:11.721273 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:11.721273 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:24:11.721273 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:24:11.721273 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:24:11.721273 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:24:11.721273 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:24:11.721273 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:24:11.721273 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 12:24:11.721273 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:11.722523 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 483/953] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:24:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:11.729831 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985978c9-3d7e-472b-b77f-fa143d52c5ab None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:11.892472 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985978c9-3d7e-472b-b77f-fa143d52c5ab None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:11.931093 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d3590-4124-4f22-8b9b-5d2bc47228fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:11.932107 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d3590-4124-4f22-8b9b-5d2bc47228fa None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:11.932588 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d3590-4124-4f22-8b9b-5d2bc47228fa None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:11.933079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d3590-4124-4f22-8b9b-5d2bc47228fa None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:11.933981 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05d3590-4124-4f22-8b9b-5d2bc47228fa None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:11.934482 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05d3590-4124-4f22-8b9b-5d2bc47228fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:11.936347 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05d3590-4124-4f22-8b9b-5d2bc47228fa None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:11.974430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 471/954] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 12:24:11 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:12.011486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0577eb-e0a7-43a7-b06f-3f94772eeaad None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:12.012509 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0577eb-e0a7-43a7-b06f-3f94772eeaad None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:12.012991 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0577eb-e0a7-43a7-b06f-3f94772eeaad None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:12.014618 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 484/955] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:12.020538 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddb097-a4e9-43c6-8d67-f9fcd22b5e9d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:12.033268 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddb097-a4e9-43c6-8d67-f9fcd22b5e9d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:12.033268 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddb097-a4e9-43c6-8d67-f9fcd22b5e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:12.042044 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4cddb097-a4e9-43c6-8d67-f9fcd22b5e9d None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:12.043638 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4cddb097-a4e9-43c6-8d67-f9fcd22b5e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:12.097903 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:12.119580 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 472/956] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:12.143257 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0577eb-e0a7-43a7-b06f-3f94772eeaad None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:12.234625 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b961d29-7cb4-4af0-a57f-887014b48468 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:12.235345 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b961d29-7cb4-4af0-a57f-887014b48468 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:12.235823 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b961d29-7cb4-4af0-a57f-887014b48468 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:12.237737 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 473/957] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:24:12.243585 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c374794-77a7-405e-ab9d-fded388e6184 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:12.244160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c374794-77a7-405e-ab9d-fded388e6184 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:12.244641 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c374794-77a7-405e-ab9d-fded388e6184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:12.257459 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0577eb-e0a7-43a7-b06f-3f94772eeaad None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:12.265453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3c374794-77a7-405e-ab9d-fded388e6184 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:12.266959 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3c374794-77a7-405e-ab9d-fded388e6184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:12.298611 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935776c-d7a7-4971-924b-df802c6f56e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:12.299281 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935776c-d7a7-4971-924b-df802c6f56e5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:12.299564 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935776c-d7a7-4971-924b-df802c6f56e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:12.299838 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935776c-d7a7-4971-924b-df802c6f56e5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:12.312310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935776c-d7a7-4971-924b-df802c6f56e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:24:12.318123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-5935776c-d7a7-4971-924b-df802c6f56e5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 12:24:12.318123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:12.318123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:12.318123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:12.318123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:12.318123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:12.318123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:12.318123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:12.318123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:12.318123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:12.318123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:12.318123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:12.318123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:24:12.318123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:24:12.318123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:24:12.318123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:24:12.319135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:24:12.319135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:24:12.319135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:24:12.319135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:24:12.319135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:12.319135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:12.319135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:12.319135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:12.319135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:12.319135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:12.319135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:12.319135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:12.319135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:12.319135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:12.319135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:12.319135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:12.320556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:12.320556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:12.320556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:12.320556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:12.320556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:24:12.320556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:24:12.320556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:24:12.320556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:24:12.320556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:24:12.320556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:24:12.320556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 12:24:12.320556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:12.323442 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 485/958] 10.222.0.49 () {56 vars in 1120 bytes} [Wed Aug 10 12:24:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:12.325763 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935776c-d7a7-4971-924b-df802c6f56e5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:12.344548 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 474/959] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:24:12.365825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c374794-77a7-405e-ab9d-fded388e6184 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:12.387957 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:12.437051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935776c-d7a7-4971-924b-df802c6f56e5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:12.471524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c374794-77a7-405e-ab9d-fded388e6184 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:12.472609 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5368340-f98e-41ef-ad54-36c6f1bb7b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:12.473384 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5368340-f98e-41ef-ad54-36c6f1bb7b2d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:12.474412 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5368340-f98e-41ef-ad54-36c6f1bb7b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:12.474698 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5368340-f98e-41ef-ad54-36c6f1bb7b2d None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:12.475441 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5368340-f98e-41ef-ad54-36c6f1bb7b2d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:12.475750 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5368340-f98e-41ef-ad54-36c6f1bb7b2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:12.477110 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5368340-f98e-41ef-ad54-36c6f1bb7b2d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:12.484939 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:12.491399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 486/960] 10.222.0.49 () {56 vars in 1115 bytes} [Wed Aug 10 12:24:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:12.493041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5368340-f98e-41ef-ad54-36c6f1bb7b2d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:12.506944 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a3ad9-74a6-47c3-8760-384718af1c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:12.507662 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a3ad9-74a6-47c3-8760-384718af1c33 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:12.507944 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a3ad9-74a6-47c3-8760-384718af1c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:12.508297 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a3ad9-74a6-47c3-8760-384718af1c33 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:12.508924 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a3ad9-74a6-47c3-8760-384718af1c33 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:12.510328 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a3ad9-74a6-47c3-8760-384718af1c33 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:12.519126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-168a3ad9-74a6-47c3-8760-384718af1c33 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 12:24:12.519126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:12.519126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:24:12.519126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:24:12.519126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:24:12.519126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:24:12.519126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:24:12.519126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:12.519126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 12:24:12.519126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:12.519126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:24:12.519126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:12.519126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:12.519126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:12.519126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:12.519126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:12.519126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:12.520325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:12.520325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:12.520325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:12.520325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:12.520325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:12.520325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:12.520325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:24:12.520325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:24:12.520325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:24:12.520325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:24:12.520325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:12.520325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:12.520325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:12.520325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:12.520325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:12.520325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:12.521186 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:12.521186 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:12.521186 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:12.521186 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:12.521186 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:12.521186 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:12.521186 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:12.521186 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:12.521186 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:12.521186 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:12.521186 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:24:12.521186 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:12.521186 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 12:24:12.521186 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:12.521186 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 475/961] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:24:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:24:12.526753 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a3ad9-74a6-47c3-8760-384718af1c33 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:12.602922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5368340-f98e-41ef-ad54-36c6f1bb7b2d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:12.623490 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:12.641936 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803beec-17f7-41cb-9ac7-4bb79108029c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:12.642324 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a3ad9-74a6-47c3-8760-384718af1c33 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:12.642571 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803beec-17f7-41cb-9ac7-4bb79108029c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:12.642850 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803beec-17f7-41cb-9ac7-4bb79108029c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:12.643124 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803beec-17f7-41cb-9ac7-4bb79108029c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:12.654740 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803beec-17f7-41cb-9ac7-4bb79108029c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:12.656208 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803beec-17f7-41cb-9ac7-4bb79108029c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:12.670953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-3803beec-17f7-41cb-9ac7-4bb79108029c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:12.670953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:12.670953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:12.670953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:12.670953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:12.670953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:12.670953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:12.670953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:12.670953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:12.670953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:12.670953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:12.670953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:12.670953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:24:12.670953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:24:12.670953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:24:12.670953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:24:12.672068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:12.672068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:12.672068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:12.672068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:12.672068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:12.672068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:12.672068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:12.672068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:12.672068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:12.672068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:12.672068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:12.672068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:12.672068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:12.672068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:12.672068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:12.672068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:12.672068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:24:12.673077 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:24:12.673077 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:12.673077 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:12.673077 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 487/962] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:24:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:12.677569 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803beec-17f7-41cb-9ac7-4bb79108029c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:12.682430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b4960-d8d5-44a5-a683-ef471126286e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:12.683037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b4960-d8d5-44a5-a683-ef471126286e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:12.683355 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b4960-d8d5-44a5-a683-ef471126286e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:12.683679 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b4960-d8d5-44a5-a683-ef471126286e None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:12.684304 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b4960-d8d5-44a5-a683-ef471126286e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:12.684604 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b4960-d8d5-44a5-a683-ef471126286e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:12.685976 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b4960-d8d5-44a5-a683-ef471126286e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:12.696600 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 476/963] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 10 12:24:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:12.697397 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b4960-d8d5-44a5-a683-ef471126286e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:12.748739 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:12.789881 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803beec-17f7-41cb-9ac7-4bb79108029c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:12.815202 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b4960-d8d5-44a5-a683-ef471126286e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:12.826056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f91862-3f70-4e27-8593-aead0db0d9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:12.826666 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f91862-3f70-4e27-8593-aead0db0d9f3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:12.826954 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f91862-3f70-4e27-8593-aead0db0d9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:12.827326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f91862-3f70-4e27-8593-aead0db0d9f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:12.827965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f91862-3f70-4e27-8593-aead0db0d9f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:12.828270 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f91862-3f70-4e27-8593-aead0db0d9f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:12.829630 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f91862-3f70-4e27-8593-aead0db0d9f3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:12.842640 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 488/964] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:12.847969 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f91862-3f70-4e27-8593-aead0db0d9f3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:12.856466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04bef7-f1d2-4c2d-bee2-d4b7fbcb6a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:12.857075 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04bef7-f1d2-4c2d-bee2-d4b7fbcb6a65 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:12.857689 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04bef7-f1d2-4c2d-bee2-d4b7fbcb6a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:12.857689 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04bef7-f1d2-4c2d-bee2-d4b7fbcb6a65 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:12.858221 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd04bef7-f1d2-4c2d-bee2-d4b7fbcb6a65 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:12.858520 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd04bef7-f1d2-4c2d-bee2-d4b7fbcb6a65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:12.860026 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd04bef7-f1d2-4c2d-bee2-d4b7fbcb6a65 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:12.864995 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 477/965] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 10 12:24:12 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:12.884350 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:12.965095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f91862-3f70-4e27-8593-aead0db0d9f3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:13.002455 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e603dd23-ae28-4a81-9fe8-d1892272f7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:13.003553 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603dd23-ae28-4a81-9fe8-d1892272f7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:13.004054 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603dd23-ae28-4a81-9fe8-d1892272f7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:13.004524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603dd23-ae28-4a81-9fe8-d1892272f7e0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:13.013824 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e603dd23-ae28-4a81-9fe8-d1892272f7e0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:13.016094 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e603dd23-ae28-4a81-9fe8-d1892272f7e0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:13.030161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-e603dd23-ae28-4a81-9fe8-d1892272f7e0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 12:24:13.030161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:13.030161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:13.030161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:13.030161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:13.030161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:13.030161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:13.030161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:13.030161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:13.030161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:13.030161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:13.030161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:13.030161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:24:13.030161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:24:13.030161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:24:13.030161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:24:13.031260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:13.031260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:13.031260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:24:13.031260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:24:13.031260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:24:13.031260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:24:13.031260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:13.031260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:13.031260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:13.031260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:13.031260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:13.031260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:13.031260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:13.031260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:13.031260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:13.031260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:13.031260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:13.031260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:13.032235 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:13.032235 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:13.032235 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:24:13.032235 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:24:13.032235 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:24:13.032235 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:24:13.032235 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:24:13.032235 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:24:13.032235 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 12:24:13.032235 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:13.032235 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 489/966] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 10 12:24:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:13.037676 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04bef7-f1d2-4c2d-bee2-d4b7fbcb6a65 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:13.141175 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04bef7-f1d2-4c2d-bee2-d4b7fbcb6a65 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:13.143540 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:13.162947 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc29ce2-2baf-48a7-b8e1-338bb3e1ee87 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:13.163348 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc29ce2-2baf-48a7-b8e1-338bb3e1ee87 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:13.163536 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc29ce2-2baf-48a7-b8e1-338bb3e1ee87 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:13.165123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 490/967] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:24:13.170146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d425c-026f-4da9-83e2-62093f6194d5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:13.170504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d425c-026f-4da9-83e2-62093f6194d5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:13.170697 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d425c-026f-4da9-83e2-62093f6194d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:13.180023 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c94510-e677-4067-90c1-bda1af4c0576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:13.180641 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c94510-e677-4067-90c1-bda1af4c0576 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:13.180921 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c94510-e677-4067-90c1-bda1af4c0576 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:13.181193 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c94510-e677-4067-90c1-bda1af4c0576 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:13.181920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c94510-e677-4067-90c1-bda1af4c0576 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:13.182228 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c94510-e677-4067-90c1-bda1af4c0576 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:13.183677 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c94510-e677-4067-90c1-bda1af4c0576 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:13.189159 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-af5d425c-026f-4da9-83e2-62093f6194d5 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:13.190340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-af5d425c-026f-4da9-83e2-62093f6194d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:13.203486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 478/968] 10.222.0.49 () {56 vars in 1149 bytes} [Wed Aug 10 12:24:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:13.208806 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c94510-e677-4067-90c1-bda1af4c0576 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:13.270372 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 491/969] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:24:13.296439 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d425c-026f-4da9-83e2-62093f6194d5 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:13.317953 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c94510-e677-4067-90c1-bda1af4c0576 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:13.322624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:13.355739 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11db180-d086-472e-b428-3db23b431402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:13.356353 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11db180-d086-472e-b428-3db23b431402 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:13.356633 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11db180-d086-472e-b428-3db23b431402 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:13.356906 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11db180-d086-472e-b428-3db23b431402 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:13.389033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:13.412665 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d425c-026f-4da9-83e2-62093f6194d5 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:13.428188 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11db180-d086-472e-b428-3db23b431402 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:13.430193 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11db180-d086-472e-b428-3db23b431402 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:13.454565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb27028-89c8-4497-a93f-a67724ef88ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:13.455224 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb27028-89c8-4497-a93f-a67724ef88ca None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:13.455508 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb27028-89c8-4497-a93f-a67724ef88ca None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:13.455782 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb27028-89c8-4497-a93f-a67724ef88ca None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:13.456357 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb27028-89c8-4497-a93f-a67724ef88ca None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:13.457705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb27028-89c8-4497-a93f-a67724ef88ca None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:13.466480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-cbb27028-89c8-4497-a93f-a67724ef88ca None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:24:13.466480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:13.466480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:24:13.466480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:24:13.466480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:24:13.466480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:24:13.466480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:24:13.466480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:13.466480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:24:13.466480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:13.466480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:24:13.466480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:13.466480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:13.466480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:13.466480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:13.466480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:13.466480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:13.467705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:13.467705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:13.467705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:13.467705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:13.467705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:13.467705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:13.467705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:24:13.467705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:24:13.467705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:24:13.467705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:24:13.467705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:13.467705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:13.467705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:13.467705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:13.467705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:13.467705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:13.468561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:13.468561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:13.468561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:13.468561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:13.468561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:13.468561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:13.468561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:13.468561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:13.468561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:13.468561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:13.468561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:24:13.468561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:13.468561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:24:13.468561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:13.472755 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 492/970] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 10 12:24:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:24:13.473431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb27028-89c8-4497-a93f-a67724ef88ca None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:13.535138 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 479/971] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:24:13 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 12:24:13.540386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11db180-d086-472e-b428-3db23b431402 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:13.570494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:13.587264 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb27028-89c8-4497-a93f-a67724ef88ca None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:13.605714 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:13.623593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a5c51-d28b-4a5a-b181-ba172ccd3642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:13.624204 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a5c51-d28b-4a5a-b181-ba172ccd3642 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:13.624481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a5c51-d28b-4a5a-b181-ba172ccd3642 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:13.624749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a5c51-d28b-4a5a-b181-ba172ccd3642 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:13.625386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a5c51-d28b-4a5a-b181-ba172ccd3642 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:13.625762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a5c51-d28b-4a5a-b181-ba172ccd3642 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:13.627278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a5c51-d28b-4a5a-b181-ba172ccd3642 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:13.633549 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 493/972] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 10 12:24:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:13.638863 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a5c51-d28b-4a5a-b181-ba172ccd3642 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:13.650848 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11db180-d086-472e-b428-3db23b431402 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:13.689717 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d8b2f-35da-42fe-9bbc-179e7168cb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:13.690343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d8b2f-35da-42fe-9bbc-179e7168cb83 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:13.690622 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d8b2f-35da-42fe-9bbc-179e7168cb83 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:13.690896 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d8b2f-35da-42fe-9bbc-179e7168cb83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:13.702059 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3d8b2f-35da-42fe-9bbc-179e7168cb83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:13.703460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3d8b2f-35da-42fe-9bbc-179e7168cb83 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:13.717253 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-1a3d8b2f-35da-42fe-9bbc-179e7168cb83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:13.717253 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:13.717253 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:13.717253 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:13.717253 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:13.717253 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:13.717253 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:13.717253 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:13.717253 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:13.717253 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:13.717253 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:13.717253 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:13.717253 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:24:13.717253 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:24:13.717253 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:24:13.717253 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:24:13.718219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:13.718219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:13.718219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:13.718219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:13.718219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:13.718219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:13.718219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:13.718219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:13.718219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:13.718219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:13.718219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:13.718219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:13.718219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:13.718219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:13.718219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:13.718219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:13.718219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:24:13.719232 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:24:13.719232 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:13.719232 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:13.719232 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 480/973] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:24:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:24:13.723963 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d8b2f-35da-42fe-9bbc-179e7168cb83 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:13.748331 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a5c51-d28b-4a5a-b181-ba172ccd3642 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:13.784970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ccc245-2ed5-4a32-b181-5f5dc5a0040a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:13.785643 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ccc245-2ed5-4a32-b181-5f5dc5a0040a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:13.785866 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ccc245-2ed5-4a32-b181-5f5dc5a0040a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:13.786142 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ccc245-2ed5-4a32-b181-5f5dc5a0040a None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:13.786764 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ccc245-2ed5-4a32-b181-5f5dc5a0040a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:13.787068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ccc245-2ed5-4a32-b181-5f5dc5a0040a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:13.788983 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ccc245-2ed5-4a32-b181-5f5dc5a0040a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:13.794005 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 494/974] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 10 12:24:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:13.799395 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ccc245-2ed5-4a32-b181-5f5dc5a0040a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:13.814916 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:13.828015 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d8b2f-35da-42fe-9bbc-179e7168cb83 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:13.842970 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:13.863731 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd41d402-00f3-4953-8869-e33658580187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:13.864437 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41d402-00f3-4953-8869-e33658580187 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:13.864659 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41d402-00f3-4953-8869-e33658580187 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:13.864933 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41d402-00f3-4953-8869-e33658580187 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:13.865613 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd41d402-00f3-4953-8869-e33658580187 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:13.866044 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd41d402-00f3-4953-8869-e33658580187 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:13.867695 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd41d402-00f3-4953-8869-e33658580187 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:13.882272 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 481/975] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:13.887885 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd41d402-00f3-4953-8869-e33658580187 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:13.913162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ccc245-2ed5-4a32-b181-5f5dc5a0040a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:13.950091 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a79ffc-fee9-4a3d-b319-6e9a405bf6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:13.951079 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a79ffc-fee9-4a3d-b319-6e9a405bf6ed None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:13.951589 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a79ffc-fee9-4a3d-b319-6e9a405bf6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:13.952063 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a79ffc-fee9-4a3d-b319-6e9a405bf6ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:13.952939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a79ffc-fee9-4a3d-b319-6e9a405bf6ed None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:13.954558 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a79ffc-fee9-4a3d-b319-6e9a405bf6ed None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:13.990149 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd41d402-00f3-4953-8869-e33658580187 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:14.025492 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b278e41-f299-4d07-b4b3-dc8d5c141efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:14.026125 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b278e41-f299-4d07-b4b3-dc8d5c141efc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:14.026408 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b278e41-f299-4d07-b4b3-dc8d5c141efc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:14.026691 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b278e41-f299-4d07-b4b3-dc8d5c141efc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:14.033409 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b278e41-f299-4d07-b4b3-dc8d5c141efc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:14.034928 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b278e41-f299-4d07-b4b3-dc8d5c141efc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:14.046264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-0b278e41-f299-4d07-b4b3-dc8d5c141efc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:24:14.046264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:14.046264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:14.046264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:14.046264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:14.046264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:14.046264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:14.046264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:14.046264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:14.046264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:14.046264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:14.046264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:14.046264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:24:14.046264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:24:14.046264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:24:14.047734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:24:14.047734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:14.047734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:14.047734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:14.047734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:14.047734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:14.047734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:14.047734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:14.047734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:14.047734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:14.047734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:14.047734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:14.047734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:14.047734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:14.047734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:14.047734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:14.047734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:14.048670 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:24:14.048670 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:24:14.048670 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:24:14.048670 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:24:14.048670 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:24:14.048670 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:24:14.048670 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:24:14.048670 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:14.051477 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 482/976] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:24:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:24:14.053047 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b278e41-f299-4d07-b4b3-dc8d5c141efc None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:14.067437 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 495/977] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:24:13 2022] POST /identity/v3/endpoints => generated 349 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:24:14.099557 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:14.173124 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b278e41-f299-4d07-b4b3-dc8d5c141efc None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:14.217140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d06f8-a642-45a5-b186-d83e9ae775b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:14.217760 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d06f8-a642-45a5-b186-d83e9ae775b4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:14.218049 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d06f8-a642-45a5-b186-d83e9ae775b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:14.218324 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d06f8-a642-45a5-b186-d83e9ae775b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:14.219018 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d06f8-a642-45a5-b186-d83e9ae775b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:14.219361 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d06f8-a642-45a5-b186-d83e9ae775b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:14.220807 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d06f8-a642-45a5-b186-d83e9ae775b4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:14.233914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 483/978] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:24:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:14.240048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a79ffc-fee9-4a3d-b319-6e9a405bf6ed None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:14.318320 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:14.352803 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a79ffc-fee9-4a3d-b319-6e9a405bf6ed None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:14.388000 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b827b-a8dc-4925-821d-3dee109bf430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:14.388657 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b827b-a8dc-4925-821d-3dee109bf430 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:14.389319 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b827b-a8dc-4925-821d-3dee109bf430 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:14.389948 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b827b-a8dc-4925-821d-3dee109bf430 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:14.391346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b827b-a8dc-4925-821d-3dee109bf430 None admin] RBAC: Adding query filter params (role.id=eb8debe34fab43f683a44c57759db243, scope.project.id=8e201e5f281a4977be503e22daf149d7, user.id=e42059e9af9842548ea08dcd39ff0b6e) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:14.391661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b827b-a8dc-4925-821d-3dee109bf430 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:14.393090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b827b-a8dc-4925-821d-3dee109bf430 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:14.407206 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 496/979] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:24:14 2022] GET /identity/v3/role_assignments?user.id=e42059e9af9842548ea08dcd39ff0b6e&scope.project.id=8e201e5f281a4977be503e22daf149d7&role.id=eb8debe34fab43f683a44c57759db243 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:14.892529 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab34ab-b890-4222-9e8b-a03ad5645901 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:14.892904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab34ab-b890-4222-9e8b-a03ad5645901 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:14.893094 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab34ab-b890-4222-9e8b-a03ad5645901 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:14.894464 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 484/980] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:14.900004 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b47673-c01b-4e16-8f2f-206d5493eed5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:14.900300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b47673-c01b-4e16-8f2f-206d5493eed5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:14.900578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b47673-c01b-4e16-8f2f-206d5493eed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:14.919583 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d4b47673-c01b-4e16-8f2f-206d5493eed5 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:14.920785 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d4b47673-c01b-4e16-8f2f-206d5493eed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:14.989944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 497/981] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:15.012617 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdab34ab-b890-4222-9e8b-a03ad5645901 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:15.107276 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:15.119081 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdab34ab-b890-4222-9e8b-a03ad5645901 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:15.154393 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f852bb-dab6-48be-8655-0dc13d33adda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:15.154922 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f852bb-dab6-48be-8655-0dc13d33adda None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:15.155224 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f852bb-dab6-48be-8655-0dc13d33adda None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:15.155501 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f852bb-dab6-48be-8655-0dc13d33adda None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:15.156151 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f852bb-dab6-48be-8655-0dc13d33adda None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:15.157511 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f852bb-dab6-48be-8655-0dc13d33adda None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:15.259500 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 485/982] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 10 12:24:15 2022] POST /identity/v3/services => generated 242 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:24:16.091549 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716b3f2-3272-4ed8-ad39-b799cfb7e1e2 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:16.092070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716b3f2-3272-4ed8-ad39-b799cfb7e1e2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:16.092070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716b3f2-3272-4ed8-ad39-b799cfb7e1e2 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:16.093464 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 498/983] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:16.098579 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad922b-2a09-4706-bef7-a7a97ab84223 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:16.098864 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad922b-2a09-4706-bef7-a7a97ab84223 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:16.099136 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad922b-2a09-4706-bef7-a7a97ab84223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:16.118167 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6cad922b-2a09-4706-bef7-a7a97ab84223 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:16.119415 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6cad922b-2a09-4706-bef7-a7a97ab84223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:16.192345 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 486/984] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:16 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:16.215042 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716b3f2-3272-4ed8-ad39-b799cfb7e1e2 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:16.286470 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:16.325569 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716b3f2-3272-4ed8-ad39-b799cfb7e1e2 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:16.361248 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ead1d-e07b-4d11-b2f5-16a240d9f3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:16.361793 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ead1d-e07b-4d11-b2f5-16a240d9f3b2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:16.362069 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ead1d-e07b-4d11-b2f5-16a240d9f3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:16.362338 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ead1d-e07b-4d11-b2f5-16a240d9f3b2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:16.362924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ead1d-e07b-4d11-b2f5-16a240d9f3b2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:16.364306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ead1d-e07b-4d11-b2f5-16a240d9f3b2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:16.373607 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-ad2ead1d-e07b-4d11-b2f5-16a240d9f3b2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:24:16.373607 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:16.373607 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:24:16.373607 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:24:16.373607 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:24:16.373607 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:24:16.373607 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:24:16.373607 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:16.373607 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:24:16.373607 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:16.373607 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:24:16.373607 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:16.373607 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:16.373607 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:16.373607 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:16.373607 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:16.373607 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:16.374661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:16.374661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:16.374661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:16.374661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:16.374661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:16.374661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:16.374661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:24:16.374661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:24:16.374661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:24:16.374661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:24:16.374661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:16.374661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:16.374661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:16.374661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:16.374661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:16.374661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:16.375923 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:16.375923 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:16.375923 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:16.375923 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:16.375923 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:16.375923 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:16.375923 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:16.375923 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:16.375923 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:16.375923 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:16.375923 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:24:16.375923 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:16.375923 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:24:16.375923 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:16.379649 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 499/985] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 10 12:24:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:16.381090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad922b-2a09-4706-bef7-a7a97ab84223 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:16.441172 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c7670-ac0e-4046-89f3-7a347109dce1 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:16.441465 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c7670-ac0e-4046-89f3-7a347109dce1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:16.441713 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c7670-ac0e-4046-89f3-7a347109dce1 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:16.443227 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 500/986] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:24:16.448396 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa1710-a15f-49a8-893c-ff9c6fad7a95 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:16.448682 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa1710-a15f-49a8-893c-ff9c6fad7a95 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:16.448955 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa1710-a15f-49a8-893c-ff9c6fad7a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:16.466291 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:16.466690 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b1aa1710-a15f-49a8-893c-ff9c6fad7a95 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:16.467923 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b1aa1710-a15f-49a8-893c-ff9c6fad7a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:16.492241 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad922b-2a09-4706-bef7-a7a97ab84223 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:16.534367 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ff194-60d5-44d4-9ecd-9c1a718d8a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:16.534971 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ff194-60d5-44d4-9ecd-9c1a718d8a4b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:16.535277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ff194-60d5-44d4-9ecd-9c1a718d8a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:16.535549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ff194-60d5-44d4-9ecd-9c1a718d8a4b None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:16.536184 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ff194-60d5-44d4-9ecd-9c1a718d8a4b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:16.536486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ff194-60d5-44d4-9ecd-9c1a718d8a4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:16.538382 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ff194-60d5-44d4-9ecd-9c1a718d8a4b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:16.541465 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 501/987] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:16 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:24:16.547954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 487/988] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 10 12:24:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:16.549505 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa1710-a15f-49a8-893c-ff9c6fad7a95 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:16.562516 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ff194-60d5-44d4-9ecd-9c1a718d8a4b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:16.609024 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:16.655239 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa1710-a15f-49a8-893c-ff9c6fad7a95 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:16.668884 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ff194-60d5-44d4-9ecd-9c1a718d8a4b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:16.693294 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faeab06-f400-41fc-8f98-f5eda5825ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:16.694321 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faeab06-f400-41fc-8f98-f5eda5825ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:16.694884 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faeab06-f400-41fc-8f98-f5eda5825ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:16.695167 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faeab06-f400-41fc-8f98-f5eda5825ee9 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:16.696075 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faeab06-f400-41fc-8f98-f5eda5825ee9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:16.696386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faeab06-f400-41fc-8f98-f5eda5825ee9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:16.698433 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faeab06-f400-41fc-8f98-f5eda5825ee9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:16.704691 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c307246-e37c-4509-adfa-a8bb2097e3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:16.705326 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c307246-e37c-4509-adfa-a8bb2097e3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:16.705612 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c307246-e37c-4509-adfa-a8bb2097e3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:16.705883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c307246-e37c-4509-adfa-a8bb2097e3b8 None admin] PATH_INFO: `/v3/services/image` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:16.706459 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c307246-e37c-4509-adfa-a8bb2097e3b8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:16.707853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c307246-e37c-4509-adfa-a8bb2097e3b8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:16.709272 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 502/989] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 10 12:24:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:16.710944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faeab06-f400-41fc-8f98-f5eda5825ee9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:16.717350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-7c307246-e37c-4509-adfa-a8bb2097e3b8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 12:24:16.717350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:16.717350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:24:16.717350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:24:16.717350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:24:16.717350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:24:16.717350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:24:16.717350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:16.717350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 12:24:16.717350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:16.717350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:24:16.717350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:16.717350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:16.717350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:16.717350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:16.717350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:16.717350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:16.718369 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:16.718369 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:16.718369 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:16.718369 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:16.718369 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:16.718369 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:16.718369 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:24:16.718369 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:24:16.718369 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:24:16.718369 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:24:16.718369 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:16.718369 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:16.718369 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:16.718369 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:16.718369 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:16.718369 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:16.719244 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:16.719244 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:16.719244 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:16.719244 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:16.719244 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:16.719244 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:16.719244 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:16.719244 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:16.719244 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:16.719244 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:16.719244 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:24:16.719244 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:16.719244 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 12:24:16.719244 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:16.723323 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 488/990] 10.222.0.49 () {56 vars in 1105 bytes} [Wed Aug 10 12:24:16 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:24:16.724131 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c307246-e37c-4509-adfa-a8bb2097e3b8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:16.728352 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:16.818453 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faeab06-f400-41fc-8f98-f5eda5825ee9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:16.836196 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c307246-e37c-4509-adfa-a8bb2097e3b8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:16.842632 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:16.859083 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8617d72-6113-4a74-96ec-b968b50c37ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:16.859731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8617d72-6113-4a74-96ec-b968b50c37ed None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:16.860008 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8617d72-6113-4a74-96ec-b968b50c37ed None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:16.860282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8617d72-6113-4a74-96ec-b968b50c37ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:16.860986 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8617d72-6113-4a74-96ec-b968b50c37ed None admin] RBAC: Adding query filter params (interface=admin, service_id=8c82165e46d34ab5b63f4d2741cd059e, region_id=RegionOne) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:16.861296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8617d72-6113-4a74-96ec-b968b50c37ed None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:16.862643 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8617d72-6113-4a74-96ec-b968b50c37ed None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:16.873004 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 503/991] 10.222.0.49 () {56 vars in 1249 bytes} [Wed Aug 10 12:24:16 2022] GET /identity/v3/endpoints?service_id=8c82165e46d34ab5b63f4d2741cd059e&interface=admin®ion_id=RegionOne => generated 193 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:16.874098 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8617d72-6113-4a74-96ec-b968b50c37ed None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:16.876725 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43bd8a-49d7-4329-9662-7ca3ddb7ce36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:16.877339 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43bd8a-49d7-4329-9662-7ca3ddb7ce36 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:16.878303 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43bd8a-49d7-4329-9662-7ca3ddb7ce36 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:16.878303 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43bd8a-49d7-4329-9662-7ca3ddb7ce36 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:16.878555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43bd8a-49d7-4329-9662-7ca3ddb7ce36 None admin] RBAC: Adding query filter params (name=image) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:16.878797 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43bd8a-49d7-4329-9662-7ca3ddb7ce36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:16.880228 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43bd8a-49d7-4329-9662-7ca3ddb7ce36 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:16.885686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 489/992] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:24:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:16.891331 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43bd8a-49d7-4329-9662-7ca3ddb7ce36 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:16.963032 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:16.994702 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8617d72-6113-4a74-96ec-b968b50c37ed None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:17.001118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43bd8a-49d7-4329-9662-7ca3ddb7ce36 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:17.031954 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29200e2f-e732-44a6-8944-2c4794d70183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:17.032566 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29200e2f-e732-44a6-8944-2c4794d70183 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:17.032844 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29200e2f-e732-44a6-8944-2c4794d70183 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:17.033119 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29200e2f-e732-44a6-8944-2c4794d70183 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:17.033710 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29200e2f-e732-44a6-8944-2c4794d70183 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:17.035102 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29200e2f-e732-44a6-8944-2c4794d70183 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:17.039273 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e1333-861a-41b4-bee4-31207c23c313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:17.040103 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e1333-861a-41b4-bee4-31207c23c313 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:17.040544 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e1333-861a-41b4-bee4-31207c23c313 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:17.041058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e1333-861a-41b4-bee4-31207c23c313 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:17.041642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 504/993] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Aug 10 12:24:16 2022] GET /identity/v3/services => generated 1727 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:24:17.050834 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e1333-861a-41b4-bee4-31207c23c313 None admin] RBAC: Adding query filter params (type=image) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:17.050834 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e1333-861a-41b4-bee4-31207c23c313 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:17.050834 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e1333-861a-41b4-bee4-31207c23c313 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:17.050834 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 490/994] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:24:16 2022] GET /identity/v3/services?type=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:17.311406 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2442e9-71e8-4929-823b-1091a316cc71 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:17.311922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2442e9-71e8-4929-823b-1091a316cc71 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:17.312003 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2442e9-71e8-4929-823b-1091a316cc71 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:17.315217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 505/995] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:17.318730 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1635b88-2afb-4ee2-ab35-1456ec103e89 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:17.319786 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1635b88-2afb-4ee2-ab35-1456ec103e89 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:17.319864 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1635b88-2afb-4ee2-ab35-1456ec103e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:17.338314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e1635b88-2afb-4ee2-ab35-1456ec103e89 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:17.339558 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e1635b88-2afb-4ee2-ab35-1456ec103e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:17.410638 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 491/996] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:17 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:17.433996 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2442e9-71e8-4929-823b-1091a316cc71 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:17.486174 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:17.540612 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2442e9-71e8-4929-823b-1091a316cc71 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:17.574843 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973c070-c505-448c-94be-064a52eb104d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:17.575508 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973c070-c505-448c-94be-064a52eb104d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:17.575790 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973c070-c505-448c-94be-064a52eb104d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:17.576066 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973c070-c505-448c-94be-064a52eb104d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:17.576646 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973c070-c505-448c-94be-064a52eb104d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:17.578011 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973c070-c505-448c-94be-064a52eb104d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:17.587527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-2973c070-c505-448c-94be-064a52eb104d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 12:24:17.587527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:17.587527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:24:17.587527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:24:17.587527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:24:17.587527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:24:17.587527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:24:17.587527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:17.587527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 12:24:17.587527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:17.587527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:24:17.587527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:17.587527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:17.587527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:17.587527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:17.587527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:17.587527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:17.588537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:17.588537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:17.588537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:17.588537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:17.588537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:17.588537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:17.588537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:24:17.588537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:24:17.588537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:24:17.588537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:24:17.588537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:17.588537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:17.588537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:17.588537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:17.588537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:17.588537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:17.589392 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:17.589392 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:17.589392 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:17.589392 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:17.589392 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:17.589392 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:17.589392 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:17.589392 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:17.589392 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:17.589392 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:17.589392 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:24:17.589392 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:17.589392 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 12:24:17.589392 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:17.590760 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 506/997] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:24:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:17.594886 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1635b88-2afb-4ee2-ab35-1456ec103e89 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:17.659835 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:17.699824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1635b88-2afb-4ee2-ab35-1456ec103e89 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:17.734590 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd54d6-8a52-4dba-b711-96370b8ff53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:17.735238 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd54d6-8a52-4dba-b711-96370b8ff53f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:17.735505 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd54d6-8a52-4dba-b711-96370b8ff53f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:17.735780 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd54d6-8a52-4dba-b711-96370b8ff53f None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:17.736427 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd54d6-8a52-4dba-b711-96370b8ff53f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:17.736735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd54d6-8a52-4dba-b711-96370b8ff53f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:17.738132 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd54d6-8a52-4dba-b711-96370b8ff53f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:17.748271 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 492/998] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 10 12:24:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:17.749419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973c070-c505-448c-94be-064a52eb104d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:17.852217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973c070-c505-448c-94be-064a52eb104d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:17.864671 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:17.886050 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307717e6-993b-40f4-bcb9-62dd797e4df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:17.886660 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307717e6-993b-40f4-bcb9-62dd797e4df9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:17.887049 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307717e6-993b-40f4-bcb9-62dd797e4df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:17.887374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307717e6-993b-40f4-bcb9-62dd797e4df9 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:17.887999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307717e6-993b-40f4-bcb9-62dd797e4df9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:17.888305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307717e6-993b-40f4-bcb9-62dd797e4df9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:17.889658 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307717e6-993b-40f4-bcb9-62dd797e4df9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:17.899845 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 507/999] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 10 12:24:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:17.900804 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd54d6-8a52-4dba-b711-96370b8ff53f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:18.000628 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd54d6-8a52-4dba-b711-96370b8ff53f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:18.025962 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:18.035051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be4b4b-0bdb-4985-bc59-067fc4ff2c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:18.035700 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be4b4b-0bdb-4985-bc59-067fc4ff2c47 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:18.035984 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be4b4b-0bdb-4985-bc59-067fc4ff2c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:18.036259 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be4b4b-0bdb-4985-bc59-067fc4ff2c47 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:18.036960 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be4b4b-0bdb-4985-bc59-067fc4ff2c47 None admin] RBAC: Adding query filter params (interface=public, service_id=fab3f741fb054b3cb349f9140c7688c6, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:18.037272 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be4b4b-0bdb-4985-bc59-067fc4ff2c47 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:18.039539 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be4b4b-0bdb-4985-bc59-067fc4ff2c47 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:18.049081 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 493/1000] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Aug 10 12:24:17 2022] GET /identity/v3/endpoints?service_id=fab3f741fb054b3cb349f9140c7688c6&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:18.050221 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307717e6-993b-40f4-bcb9-62dd797e4df9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:18.150878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307717e6-993b-40f4-bcb9-62dd797e4df9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:18.187359 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ccf88-7d3c-4548-b4c7-b70f1b5b0248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:18.187873 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ccf88-7d3c-4548-b4c7-b70f1b5b0248 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:18.188155 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ccf88-7d3c-4548-b4c7-b70f1b5b0248 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:18.188434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ccf88-7d3c-4548-b4c7-b70f1b5b0248 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:18.189039 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50ccf88-7d3c-4548-b4c7-b70f1b5b0248 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:18.190404 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50ccf88-7d3c-4548-b4c7-b70f1b5b0248 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:18.203236 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 508/1001] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Aug 10 12:24:18 2022] GET /identity/v3/services => generated 1727 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:24:19.091466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7176f-567a-4fb0-b810-92db93e39716 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:19.091992 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7176f-567a-4fb0-b810-92db93e39716 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:19.091992 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7176f-567a-4fb0-b810-92db93e39716 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:19.093522 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 494/1002] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:19.098331 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf60252-3e72-4108-b01f-8cc5506f75ba None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:19.098724 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf60252-3e72-4108-b01f-8cc5506f75ba None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:19.098903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf60252-3e72-4108-b01f-8cc5506f75ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:19.118966 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ebf60252-3e72-4108-b01f-8cc5506f75ba None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:19.120956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ebf60252-3e72-4108-b01f-8cc5506f75ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:19.146530 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be0ce7-0c61-4e52-988a-d0a62ee353b2 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:19.146770 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be0ce7-0c61-4e52-988a-d0a62ee353b2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:19.147044 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be0ce7-0c61-4e52-988a-d0a62ee353b2 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:19.149212 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 495/1003] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:24:19.153994 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30a265-10ed-4450-965c-bbb175590862 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:19.154337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30a265-10ed-4450-965c-bbb175590862 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:19.154606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30a265-10ed-4450-965c-bbb175590862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:19.158494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:19.172629 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3f30a265-10ed-4450-965c-bbb175590862 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:19.173814 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3f30a265-10ed-4450-965c-bbb175590862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:19.196511 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 509/1004] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:19 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:19.218841 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf60252-3e72-4108-b01f-8cc5506f75ba None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:19.255320 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 496/1005] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:19 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:19.269802 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30a265-10ed-4450-965c-bbb175590862 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:19.330923 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf60252-3e72-4108-b01f-8cc5506f75ba None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:19.369762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6dadca-c9e1-4140-9896-9e53de880d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:19.370290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6dadca-c9e1-4140-9896-9e53de880d6a None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:19.370569 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6dadca-c9e1-4140-9896-9e53de880d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:19.370838 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6dadca-c9e1-4140-9896-9e53de880d6a None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:19.371761 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6dadca-c9e1-4140-9896-9e53de880d6a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:19.373142 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6dadca-c9e1-4140-9896-9e53de880d6a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:19.379612 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30a265-10ed-4450-965c-bbb175590862 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:19.400249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:19.414667 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ef88b-5ef2-47ee-a074-11597f2d097e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:19.415317 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ef88b-5ef2-47ee-a074-11597f2d097e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:19.415595 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ef88b-5ef2-47ee-a074-11597f2d097e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:19.415860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ef88b-5ef2-47ee-a074-11597f2d097e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:19.417365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ef88b-5ef2-47ee-a074-11597f2d097e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:19.418719 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ef88b-5ef2-47ee-a074-11597f2d097e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:19.427036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-3c4ef88b-5ef2-47ee-a074-11597f2d097e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:24:19.427036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:19.427036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:24:19.427036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:24:19.427036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:24:19.427036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:24:19.427036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:24:19.427036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:19.427036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:24:19.427036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:19.427036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:24:19.427036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:19.427036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:19.427036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:19.427036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:19.427036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:19.427036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:19.428209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:19.428209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:19.428209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:19.428209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:19.428209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:19.428209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:19.428209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:24:19.428209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:24:19.428209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:24:19.428209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:24:19.428209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:19.428209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:19.428209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:19.428209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:19.428209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:19.428209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:19.429487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:19.429487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:19.429487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:19.429487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:19.429487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:19.429487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:19.429487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:19.429487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:19.429487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:19.429487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:19.429487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:24:19.429487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:19.429487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:24:19.429487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:19.429487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 497/1006] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 10 12:24:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:24:19.433881 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ef88b-5ef2-47ee-a074-11597f2d097e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:19.451128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 510/1007] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 10 12:24:19 2022] POST /identity/v3/services => generated 242 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:24:19.549778 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ef88b-5ef2-47ee-a074-11597f2d097e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:19.594145 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268e815-5291-4554-832a-2653bcacbe99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:19.594760 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268e815-5291-4554-832a-2653bcacbe99 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:19.595037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268e815-5291-4554-832a-2653bcacbe99 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:19.595340 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268e815-5291-4554-832a-2653bcacbe99 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:19.595954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268e815-5291-4554-832a-2653bcacbe99 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:19.596253 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268e815-5291-4554-832a-2653bcacbe99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:19.597623 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268e815-5291-4554-832a-2653bcacbe99 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:19.603331 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 498/1008] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 10 12:24:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:19.609256 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6dadca-c9e1-4140-9896-9e53de880d6a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:19.645705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:19.717773 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6dadca-c9e1-4140-9896-9e53de880d6a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:19.754214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c91223-0ddc-4f92-b2f4-01b50a84d23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:19.754823 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c91223-0ddc-4f92-b2f4-01b50a84d23e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:19.755099 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c91223-0ddc-4f92-b2f4-01b50a84d23e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:19.755397 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c91223-0ddc-4f92-b2f4-01b50a84d23e None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:19.756043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c91223-0ddc-4f92-b2f4-01b50a84d23e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:19.756349 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c91223-0ddc-4f92-b2f4-01b50a84d23e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:19.758774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c91223-0ddc-4f92-b2f4-01b50a84d23e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:19.763909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 511/1009] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 10 12:24:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:19.769314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268e815-5291-4554-832a-2653bcacbe99 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:19.822582 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:19.877817 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268e815-5291-4554-832a-2653bcacbe99 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:19.912071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a0126-3a7c-4860-87e6-ecda45631de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:19.912600 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a0126-3a7c-4860-87e6-ecda45631de2 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:19.912943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a0126-3a7c-4860-87e6-ecda45631de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:19.913237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a0126-3a7c-4860-87e6-ecda45631de2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:19.913912 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71a0126-3a7c-4860-87e6-ecda45631de2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:19.915306 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71a0126-3a7c-4860-87e6-ecda45631de2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:19.987295 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 499/1010] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:24:19 2022] POST /identity/v3/endpoints => generated 325 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:24:20.231378 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed2d5e-65d7-406f-865d-a7ed65022f96 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:20.231866 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed2d5e-65d7-406f-865d-a7ed65022f96 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:20.231866 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed2d5e-65d7-406f-865d-a7ed65022f96 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:20.233197 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 512/1011] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:20.238366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dae1e6-d2e7-47d1-98df-fe5d722788b5 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:20.238770 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dae1e6-d2e7-47d1-98df-fe5d722788b5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:20.238959 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dae1e6-d2e7-47d1-98df-fe5d722788b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:20.260289 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a8dae1e6-d2e7-47d1-98df-fe5d722788b5 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:20.261496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a8dae1e6-d2e7-47d1-98df-fe5d722788b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:20.341001 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 500/1012] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:20 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:20.363542 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed2d5e-65d7-406f-865d-a7ed65022f96 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:20.458747 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:20.467509 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed2d5e-65d7-406f-865d-a7ed65022f96 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:20.501782 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0015315-3917-45de-b681-bf7f21ee34dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:20.502342 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0015315-3917-45de-b681-bf7f21ee34dd None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:20.502620 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0015315-3917-45de-b681-bf7f21ee34dd None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:20.502891 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0015315-3917-45de-b681-bf7f21ee34dd None admin] PATH_INFO: `/v3/services/compute` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:20.503509 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0015315-3917-45de-b681-bf7f21ee34dd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:20.504857 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0015315-3917-45de-b681-bf7f21ee34dd None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:20.514172 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-a0015315-3917-45de-b681-bf7f21ee34dd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 12:24:20.514172 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:20.514172 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:24:20.514172 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:24:20.514172 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:24:20.514172 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:24:20.514172 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:24:20.514172 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:20.514172 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 12:24:20.514172 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:20.514172 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:24:20.514172 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:20.514172 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:20.514172 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:20.514172 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:20.514172 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:20.514172 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:20.515146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:20.515146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:20.515146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:20.515146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:20.515146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:20.515146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:20.515146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:24:20.515146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:24:20.515146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:24:20.515146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:24:20.515146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:20.515146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:20.515146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:20.515146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:20.515146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:20.515146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:20.516030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:20.516030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:20.516030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:20.516030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:20.516030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:20.516030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:20.516030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:20.516030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:20.516030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:20.516030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:20.516030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:24:20.516030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:20.516030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 12:24:20.516030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:20.523218 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 513/1013] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:24:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:20.523358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dae1e6-d2e7-47d1-98df-fe5d722788b5 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:20.553226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:20.627726 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dae1e6-d2e7-47d1-98df-fe5d722788b5 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:20.663321 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4665eb-aa22-46be-8e3f-b6571f8817c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:20.664269 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4665eb-aa22-46be-8e3f-b6571f8817c6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:20.664751 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4665eb-aa22-46be-8e3f-b6571f8817c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:20.665219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4665eb-aa22-46be-8e3f-b6571f8817c6 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:20.666175 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4665eb-aa22-46be-8e3f-b6571f8817c6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:20.666678 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4665eb-aa22-46be-8e3f-b6571f8817c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:20.668305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4665eb-aa22-46be-8e3f-b6571f8817c6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:20.673985 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 501/1014] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 10 12:24:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:20.680274 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0015315-3917-45de-b681-bf7f21ee34dd None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:20.783231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0015315-3917-45de-b681-bf7f21ee34dd None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:20.816507 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c19789-5c42-4a72-801e-45a6c69c4f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:20.817110 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c19789-5c42-4a72-801e-45a6c69c4f42 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:20.817391 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c19789-5c42-4a72-801e-45a6c69c4f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:20.817720 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c19789-5c42-4a72-801e-45a6c69c4f42 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:20.818343 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c19789-5c42-4a72-801e-45a6c69c4f42 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:20.818648 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c19789-5c42-4a72-801e-45a6c69c4f42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:20.820575 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c19789-5c42-4a72-801e-45a6c69c4f42 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:20.830010 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 514/1015] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 10 12:24:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:20.831248 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4665eb-aa22-46be-8e3f-b6571f8817c6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:20.926922 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:20.934482 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4665eb-aa22-46be-8e3f-b6571f8817c6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:20.969770 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6b6bf-aca6-462a-844c-6c3e2733c138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:20.970382 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6b6bf-aca6-462a-844c-6c3e2733c138 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:20.970663 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6b6bf-aca6-462a-844c-6c3e2733c138 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:20.970935 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6b6bf-aca6-462a-844c-6c3e2733c138 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:20.971684 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb6b6bf-aca6-462a-844c-6c3e2733c138 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:20.973047 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb6b6bf-aca6-462a-844c-6c3e2733c138 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:21.071692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 502/1016] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:24:20 2022] POST /identity/v3/endpoints => generated 334 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:24:21.505103 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1518c-fb7f-4139-9b66-5d7bd1d6ed09 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:21.505602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1518c-fb7f-4139-9b66-5d7bd1d6ed09 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:21.505602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1518c-fb7f-4139-9b66-5d7bd1d6ed09 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:21.507342 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 515/1017] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:21.512304 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d648c6-a404-422b-9108-dfb2b085f90e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:21.512693 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d648c6-a404-422b-9108-dfb2b085f90e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:21.512928 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d648c6-a404-422b-9108-dfb2b085f90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:21.531278 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-32d648c6-a404-422b-9108-dfb2b085f90e None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:21.532476 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-32d648c6-a404-422b-9108-dfb2b085f90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:21.606810 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 503/1018] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:21.629275 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1518c-fb7f-4139-9b66-5d7bd1d6ed09 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:21.656478 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:21.734653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1518c-fb7f-4139-9b66-5d7bd1d6ed09 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:21.769674 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c35af3-f282-40a7-a0ed-d0966a23339b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:21.770225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c35af3-f282-40a7-a0ed-d0966a23339b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:21.770519 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c35af3-f282-40a7-a0ed-d0966a23339b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:21.770827 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c35af3-f282-40a7-a0ed-d0966a23339b None admin] PATH_INFO: `/v3/services/image` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:21.771438 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c35af3-f282-40a7-a0ed-d0966a23339b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:21.772878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c35af3-f282-40a7-a0ed-d0966a23339b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:21.781617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-f6c35af3-f282-40a7-a0ed-d0966a23339b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 12:24:21.781617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:21.781617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:24:21.781617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:24:21.781617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:24:21.781617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:24:21.781617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:24:21.781617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:21.781617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 12:24:21.781617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:21.781617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:24:21.781617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:21.781617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:21.781617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:21.781617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:21.781617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:21.781617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:21.782672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:21.782672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:21.782672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:21.782672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:21.782672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:21.782672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:21.782672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:24:21.782672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:24:21.782672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:24:21.782672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:24:21.782672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:21.782672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:21.782672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:21.782672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:21.782672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:21.782672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:21.783551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:21.783551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:21.783551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:21.783551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:21.783551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:21.783551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:21.783551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:21.783551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:21.783551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:21.783551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:21.783551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:24:21.783551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:21.783551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 12:24:21.783551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:21.788370 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 516/1019] 10.222.0.49 () {56 vars in 1105 bytes} [Wed Aug 10 12:24:21 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:21.789492 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d648c6-a404-422b-9108-dfb2b085f90e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:21.869504 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:21.893368 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d648c6-a404-422b-9108-dfb2b085f90e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:21.927336 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53c0ff-cc5f-4da7-be87-46922ee668de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:21.927951 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53c0ff-cc5f-4da7-be87-46922ee668de None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:21.928231 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53c0ff-cc5f-4da7-be87-46922ee668de None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:21.928555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53c0ff-cc5f-4da7-be87-46922ee668de None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:21.929417 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53c0ff-cc5f-4da7-be87-46922ee668de None admin] RBAC: Adding query filter params (name=image) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:21.929723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53c0ff-cc5f-4da7-be87-46922ee668de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:21.931091 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53c0ff-cc5f-4da7-be87-46922ee668de None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:21.941317 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 504/1020] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:24:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:21.942458 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c35af3-f282-40a7-a0ed-d0966a23339b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:21.998636 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93080bdc-b96c-4924-bbf7-ff88ac8e3c36 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:21.999002 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93080bdc-b96c-4924-bbf7-ff88ac8e3c36 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:21.999199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93080bdc-b96c-4924-bbf7-ff88ac8e3c36 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:22.000534 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 505/1021] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:24:22.005355 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cecb5e-9d96-424d-bc2a-a3fe73b4d841 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:22.005632 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cecb5e-9d96-424d-bc2a-a3fe73b4d841 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:22.005901 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cecb5e-9d96-424d-bc2a-a3fe73b4d841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:22.024974 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a2cecb5e-9d96-424d-bc2a-a3fe73b4d841 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:22.026090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a2cecb5e-9d96-424d-bc2a-a3fe73b4d841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:22.038686 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:22.052996 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c35af3-f282-40a7-a0ed-d0966a23339b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:22.088426 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60289e8-7ee1-40c1-b2d4-0adf7dd9656a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:22.089025 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60289e8-7ee1-40c1-b2d4-0adf7dd9656a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:22.089308 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60289e8-7ee1-40c1-b2d4-0adf7dd9656a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:22.089657 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60289e8-7ee1-40c1-b2d4-0adf7dd9656a None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:22.090200 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60289e8-7ee1-40c1-b2d4-0adf7dd9656a None admin] RBAC: Adding query filter params (type=image) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:22.090504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60289e8-7ee1-40c1-b2d4-0adf7dd9656a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:22.091894 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60289e8-7ee1-40c1-b2d4-0adf7dd9656a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:22.097196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 517/1022] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:24:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:22.102515 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 506/1023] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:24:22.102901 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60289e8-7ee1-40c1-b2d4-0adf7dd9656a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:22.125215 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cecb5e-9d96-424d-bc2a-a3fe73b4d841 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:22.198854 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:22.211472 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60289e8-7ee1-40c1-b2d4-0adf7dd9656a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:22.234594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cecb5e-9d96-424d-bc2a-a3fe73b4d841 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:22.248707 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeae7ad-9d42-4615-8a7b-206715f9dc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:22.249326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeae7ad-9d42-4615-8a7b-206715f9dc7d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:22.249601 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeae7ad-9d42-4615-8a7b-206715f9dc7d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:22.249876 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeae7ad-9d42-4615-8a7b-206715f9dc7d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:22.250573 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeae7ad-9d42-4615-8a7b-206715f9dc7d None admin] RBAC: Adding query filter params (interface=public, service_id=4f46fa904eef4b7c8974469272b906af, region_id=RegionOne) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:22.250886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeae7ad-9d42-4615-8a7b-206715f9dc7d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:22.252278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeae7ad-9d42-4615-8a7b-206715f9dc7d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:22.261010 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 518/1024] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Aug 10 12:24:22 2022] GET /identity/v3/endpoints?service_id=4f46fa904eef4b7c8974469272b906af&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:22.266241 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeae7ad-9d42-4615-8a7b-206715f9dc7d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:22.278363 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3d951-871f-479f-a4c3-f2a839e64146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:22.279038 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3d951-871f-479f-a4c3-f2a839e64146 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:22.279354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3d951-871f-479f-a4c3-f2a839e64146 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:22.279625 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3d951-871f-479f-a4c3-f2a839e64146 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:22.280956 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3d951-871f-479f-a4c3-f2a839e64146 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:22.282182 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3d951-871f-479f-a4c3-f2a839e64146 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:22.282624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:22.289048 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 507/1025] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:24:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:22.290029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3d951-871f-479f-a4c3-f2a839e64146 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:22.379816 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeae7ad-9d42-4615-8a7b-206715f9dc7d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:22.399624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3d951-871f-479f-a4c3-f2a839e64146 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:22.417108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf47d1-a39a-4075-927f-002f9b2321b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:22.417718 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf47d1-a39a-4075-927f-002f9b2321b2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:22.417997 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf47d1-a39a-4075-927f-002f9b2321b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:22.418269 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf47d1-a39a-4075-927f-002f9b2321b2 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:22.418980 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf47d1-a39a-4075-927f-002f9b2321b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:22.420369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf47d1-a39a-4075-927f-002f9b2321b2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:22.432728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a280a-04a2-49b0-a58d-25cc880192a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:22.433349 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a280a-04a2-49b0-a58d-25cc880192a0 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:22.433625 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a280a-04a2-49b0-a58d-25cc880192a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:22.433893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a280a-04a2-49b0-a58d-25cc880192a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:22.434549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661a280a-04a2-49b0-a58d-25cc880192a0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:22.435401 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 519/1026] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Aug 10 12:24:22 2022] GET /identity/v3/services => generated 1957 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:24:22.436053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661a280a-04a2-49b0-a58d-25cc880192a0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:22.455306 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:22.520303 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 508/1027] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 10 12:24:22 2022] POST /identity/v3/projects => generated 311 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:24:23.224023 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0007b-c6ab-4a08-afc8-7e801ea2ac6a None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:23.224564 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0007b-c6ab-4a08-afc8-7e801ea2ac6a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:23.224564 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0007b-c6ab-4a08-afc8-7e801ea2ac6a None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:23.226667 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 520/1028] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:23.231870 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b09535-9293-432e-a8dc-9817db35003b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:23.232163 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b09535-9293-432e-a8dc-9817db35003b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:23.232439 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b09535-9293-432e-a8dc-9817db35003b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:23.251324 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-48b09535-9293-432e-a8dc-9817db35003b None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:23.252526 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-48b09535-9293-432e-a8dc-9817db35003b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:23.330244 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 509/1029] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:23.353678 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf0007b-c6ab-4a08-afc8-7e801ea2ac6a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:23.438961 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:23.465031 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf0007b-c6ab-4a08-afc8-7e801ea2ac6a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:23.503989 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47edef0-bf21-40dd-8e35-e927ba7e823e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:23.504551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47edef0-bf21-40dd-8e35-e927ba7e823e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:23.504829 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47edef0-bf21-40dd-8e35-e927ba7e823e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:23.505117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47edef0-bf21-40dd-8e35-e927ba7e823e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:23.516687 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47edef0-bf21-40dd-8e35-e927ba7e823e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:24:23.521692 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-a47edef0-bf21-40dd-8e35-e927ba7e823e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 12:24:23.521692 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:23.521692 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:23.521692 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:23.521692 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:23.521692 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:23.521692 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:23.521692 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:23.521692 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:23.521692 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:23.521692 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:23.521692 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:23.521692 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:24:23.521692 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:24:23.521692 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:24:23.521692 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:24:23.523327 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:24:23.523327 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:24:23.523327 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:24:23.523327 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:24:23.523327 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:23.523327 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:23.523327 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:23.523327 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:23.523327 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:23.523327 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:23.523327 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:23.523327 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:23.523327 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:23.523327 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:23.523327 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:23.523327 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:23.531396 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:23.531396 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:23.531396 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:23.531396 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:23.531396 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:24:23.531396 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:24:23.531396 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:24:23.531396 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:24:23.531396 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:24:23.531396 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:24:23.531396 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 12:24:23.531396 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:23.531396 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 521/1030] 10.222.0.49 () {56 vars in 1120 bytes} [Wed Aug 10 12:24:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:23.535109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b09535-9293-432e-a8dc-9817db35003b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:23.659211 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b09535-9293-432e-a8dc-9817db35003b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:23.686145 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:23.704967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e93070-ea10-4d7b-a72c-f76710dd0e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:23.705583 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e93070-ea10-4d7b-a72c-f76710dd0e05 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:23.705864 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e93070-ea10-4d7b-a72c-f76710dd0e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:23.706138 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e93070-ea10-4d7b-a72c-f76710dd0e05 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:23.706802 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e93070-ea10-4d7b-a72c-f76710dd0e05 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:23.707108 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e93070-ea10-4d7b-a72c-f76710dd0e05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:23.708514 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e93070-ea10-4d7b-a72c-f76710dd0e05 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:23.719395 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 510/1031] 10.222.0.49 () {56 vars in 1115 bytes} [Wed Aug 10 12:24:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:23.726275 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47edef0-bf21-40dd-8e35-e927ba7e823e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:23.851957 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47edef0-bf21-40dd-8e35-e927ba7e823e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:23.893386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c4bacc-6ce8-4d26-9313-ddbe5bc7793a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:23.894446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4bacc-6ce8-4d26-9313-ddbe5bc7793a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:23.894974 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4bacc-6ce8-4d26-9313-ddbe5bc7793a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:23.895564 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4bacc-6ce8-4d26-9313-ddbe5bc7793a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:23.904625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:23.910160 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c4bacc-6ce8-4d26-9313-ddbe5bc7793a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:23.912208 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c4bacc-6ce8-4d26-9313-ddbe5bc7793a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:23.931743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-48c4bacc-6ce8-4d26-9313-ddbe5bc7793a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:23.931743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:23.931743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:23.931743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:23.931743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:23.931743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:23.931743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:23.931743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:23.931743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:23.931743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:23.931743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:23.931743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:23.931743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:24:23.931743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:24:23.931743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:24:23.931743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:24:23.932749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:23.932749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:23.932749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:23.932749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:23.932749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:23.932749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:23.932749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:23.932749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:23.932749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:23.932749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:23.932749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:23.932749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:23.932749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:23.932749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:23.932749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:23.932749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:23.932749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:24:23.938487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:24:23.938487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:23.938487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:23.938487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 522/1032] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:24:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:24:23.940238 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e93070-ea10-4d7b-a72c-f76710dd0e05 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:24.058464 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e93070-ea10-4d7b-a72c-f76710dd0e05 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:24.097630 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d27d0e-664d-438d-947f-efeb6c2b85ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:24.098248 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d27d0e-664d-438d-947f-efeb6c2b85ce None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:24.098527 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d27d0e-664d-438d-947f-efeb6c2b85ce None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:24.098799 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d27d0e-664d-438d-947f-efeb6c2b85ce None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:24.099475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d27d0e-664d-438d-947f-efeb6c2b85ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:24.099781 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d27d0e-664d-438d-947f-efeb6c2b85ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:24.101431 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d27d0e-664d-438d-947f-efeb6c2b85ce None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:24.119840 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 511/1033] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:24.121100 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c4bacc-6ce8-4d26-9313-ddbe5bc7793a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:24.232769 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c4bacc-6ce8-4d26-9313-ddbe5bc7793a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:24.271237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd5d71-faad-48d3-9aa7-cd7b920e48a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:24.271881 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd5d71-faad-48d3-9aa7-cd7b920e48a8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:24.272160 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd5d71-faad-48d3-9aa7-cd7b920e48a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:24.273012 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd5d71-faad-48d3-9aa7-cd7b920e48a8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:24.285582 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd5d71-faad-48d3-9aa7-cd7b920e48a8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:24.287025 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd5d71-faad-48d3-9aa7-cd7b920e48a8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:24.298090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-bddd5d71-faad-48d3-9aa7-cd7b920e48a8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:24:24.298090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:24.298090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:24.298090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:24.298090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:24.298090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:24.298090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:24.298090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:24.298090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:24.298090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:24.298090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:24.298090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:24.298090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:24:24.298090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:24:24.298090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:24:24.298090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:24:24.299056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:24.299056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:24.299056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:24:24.299056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:24:24.299056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:24:24.299056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:24:24.299056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:24.299056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:24.299056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:24.299056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:24.299056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:24.299056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:24.299056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:24.299056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:24.299056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:24.299056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:24.299056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:24.299056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:24.300624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:24.300624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:24.300624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:24:24.300624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:24:24.300624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:24:24.300624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:24:24.300624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:24:24.300624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:24:24.300624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:24:24.300624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:24.304040 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 523/1034] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Aug 10 12:24:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:24.305508 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d27d0e-664d-438d-947f-efeb6c2b85ce None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:24.352183 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:24.409812 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d27d0e-664d-438d-947f-efeb6c2b85ce None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:24.444812 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94f534-5edc-4c6f-9e3d-3f93a8b8c3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:24.445420 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94f534-5edc-4c6f-9e3d-3f93a8b8c3f6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:24.445702 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94f534-5edc-4c6f-9e3d-3f93a8b8c3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:24.445977 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94f534-5edc-4c6f-9e3d-3f93a8b8c3f6 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:24.446700 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef94f534-5edc-4c6f-9e3d-3f93a8b8c3f6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:24.447009 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef94f534-5edc-4c6f-9e3d-3f93a8b8c3f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:24.448466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef94f534-5edc-4c6f-9e3d-3f93a8b8c3f6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:24.471210 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 512/1035] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Aug 10 12:24:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:24.473399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd5d71-faad-48d3-9aa7-cd7b920e48a8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:24.487560 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdf40a-1381-4bb5-ae3d-c812e7e6b87d None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:24.487855 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdf40a-1381-4bb5-ae3d-c812e7e6b87d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:24.488140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdf40a-1381-4bb5-ae3d-c812e7e6b87d None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:24.489753 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 513/1036] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:24.494874 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b42c3-31fe-44d4-896d-60c5690b5e62 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:24.495164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b42c3-31fe-44d4-896d-60c5690b5e62 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:24.495472 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b42c3-31fe-44d4-896d-60c5690b5e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:24.514539 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-2a4b42c3-31fe-44d4-896d-60c5690b5e62 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:24.515784 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2a4b42c3-31fe-44d4-896d-60c5690b5e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:24.535312 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:24.588625 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 514/1037] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:24:24.589885 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168e6ba-e118-45af-af8a-27dddb91bd05 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:24.590177 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168e6ba-e118-45af-af8a-27dddb91bd05 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:24.590453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168e6ba-e118-45af-af8a-27dddb91bd05 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:24.591980 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 515/1038] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:24.596697 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a6d38-9433-4ef7-898a-334d3cd98e24 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:24.596984 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a6d38-9433-4ef7-898a-334d3cd98e24 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:24.597330 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a6d38-9433-4ef7-898a-334d3cd98e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:24.604760 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd5d71-faad-48d3-9aa7-cd7b920e48a8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:24.615496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a39a6d38-9433-4ef7-898a-334d3cd98e24 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:24.617304 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a39a6d38-9433-4ef7-898a-334d3cd98e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:24.639617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd782b-4b64-424b-8d38-54e0eb66a2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:24.640320 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd782b-4b64-424b-8d38-54e0eb66a2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:24.640597 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd782b-4b64-424b-8d38-54e0eb66a2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:24.640864 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd782b-4b64-424b-8d38-54e0eb66a2b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:24.641366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:24.683475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 516/1039] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:24:24.684499 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a6d38-9433-4ef7-898a-334d3cd98e24 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:24.711747 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effd782b-4b64-424b-8d38-54e0eb66a2b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:24.713261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effd782b-4b64-424b-8d38-54e0eb66a2b5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:24.761963 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:24.783788 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a6d38-9433-4ef7-898a-334d3cd98e24 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:24.812855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 524/1040] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:24:24 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:24:24.816112 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd782b-4b64-424b-8d38-54e0eb66a2b5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:24.820330 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ebbc5c-56f9-4b7a-b1e2-f1193d704a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:24.820963 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ebbc5c-56f9-4b7a-b1e2-f1193d704a6a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:24.821243 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ebbc5c-56f9-4b7a-b1e2-f1193d704a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:24.821514 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ebbc5c-56f9-4b7a-b1e2-f1193d704a6a None admin] PATH_INFO: `/v3/services/image` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:24.822085 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ebbc5c-56f9-4b7a-b1e2-f1193d704a6a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:24.823550 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ebbc5c-56f9-4b7a-b1e2-f1193d704a6a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:24.832286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b8ebbc5c-56f9-4b7a-b1e2-f1193d704a6a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 12:24:24.832286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:24.832286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:24:24.832286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:24:24.832286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:24:24.832286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:24:24.832286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:24:24.832286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:24.832286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 12:24:24.832286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:24.832286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:24:24.832286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:24.832286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:24.832286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:24.832286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:24.832286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:24.832286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:24.833307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:24.833307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:24.833307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:24.833307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:24.833307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:24.833307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:24.833307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:24:24.833307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:24:24.833307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:24:24.833307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:24:24.833307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:24.833307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:24.833307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:24.833307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:24.833307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:24.833307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:24.834143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:24.834143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:24.834143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:24.834143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:24.834143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:24.834143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:24.834143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:24.834143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:24.834143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:24.834143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:24.834143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:24:24.834143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:24:24.834143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 12:24:24.834143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:24.834143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 517/1041] 10.222.0.49 () {56 vars in 1105 bytes} [Wed Aug 10 12:24:24 2022] GET /identity/v3/services/image => generated 86 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:24.834987 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ebbc5c-56f9-4b7a-b1e2-f1193d704a6a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:24.860776 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:24.916479 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd782b-4b64-424b-8d38-54e0eb66a2b5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:24.937316 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ebbc5c-56f9-4b7a-b1e2-f1193d704a6a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:24.949617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a482e5-af4e-43d9-a4c5-f5738d188478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:24.950252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a482e5-af4e-43d9-a4c5-f5738d188478 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:24.950531 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a482e5-af4e-43d9-a4c5-f5738d188478 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:24.950806 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a482e5-af4e-43d9-a4c5-f5738d188478 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:24.951933 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a482e5-af4e-43d9-a4c5-f5738d188478 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:24.953346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a482e5-af4e-43d9-a4c5-f5738d188478 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:24.955003 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 525/1042] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:24:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:24.955806 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a482e5-af4e-43d9-a4c5-f5738d188478 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:24.971825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71406edc-c471-4fc9-8375-028f4733998d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:24.972484 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71406edc-c471-4fc9-8375-028f4733998d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:24.972763 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71406edc-c471-4fc9-8375-028f4733998d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:24.973029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71406edc-c471-4fc9-8375-028f4733998d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:24.980933 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:24.984650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71406edc-c471-4fc9-8375-028f4733998d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:24.986138 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71406edc-c471-4fc9-8375-028f4733998d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:25.001201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-71406edc-c471-4fc9-8375-028f4733998d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:25.001201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:25.001201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:25.001201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:25.001201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:25.001201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:25.001201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:25.001201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:25.001201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:25.001201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:25.001201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:25.001201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:25.001201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:24:25.001201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:24:25.001201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:24:25.001201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:24:25.003891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:25.003891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:25.003891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:25.003891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:25.003891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:25.003891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:25.003891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:25.003891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:25.003891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:25.003891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:25.003891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:25.003891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:25.003891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:25.003891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:25.003891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:25.003891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:25.003891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:24:25.004783 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:24:25.004783 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:25.004783 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:25.004783 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 518/1043] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:24:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:25.004783 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71406edc-c471-4fc9-8375-028f4733998d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:25.063297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a482e5-af4e-43d9-a4c5-f5738d188478 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:25.083027 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:25.096898 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d2ff9-d3bd-4938-aba0-1a8a5bd54d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:25.097509 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d2ff9-d3bd-4938-aba0-1a8a5bd54d30 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:25.097780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d2ff9-d3bd-4938-aba0-1a8a5bd54d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:25.098052 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d2ff9-d3bd-4938-aba0-1a8a5bd54d30 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:25.098670 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d2ff9-d3bd-4938-aba0-1a8a5bd54d30 None admin] RBAC: Adding query filter params (name=image) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:25.098977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d2ff9-d3bd-4938-aba0-1a8a5bd54d30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:25.100372 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d2ff9-d3bd-4938-aba0-1a8a5bd54d30 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:25.105745 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 526/1044] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:24:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:25.106270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71406edc-c471-4fc9-8375-028f4733998d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:25.106527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d2ff9-d3bd-4938-aba0-1a8a5bd54d30 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:25.141231 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53511a4e-4697-42d2-9fc8-3c5bf8d93be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:25.141830 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53511a4e-4697-42d2-9fc8-3c5bf8d93be1 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:25.142112 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53511a4e-4697-42d2-9fc8-3c5bf8d93be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:25.142385 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53511a4e-4697-42d2-9fc8-3c5bf8d93be1 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:25.143018 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53511a4e-4697-42d2-9fc8-3c5bf8d93be1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:25.144498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53511a4e-4697-42d2-9fc8-3c5bf8d93be1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:25.188009 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:25.205598 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d2ff9-d3bd-4938-aba0-1a8a5bd54d30 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:25.220908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 519/1045] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 12:24:25 2022] POST /identity/v3/users => generated 285 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:24:25.221741 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53511a4e-4697-42d2-9fc8-3c5bf8d93be1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:25.241075 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66c66a-17ae-420d-af63-c60c3cbde7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:25.241808 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66c66a-17ae-420d-af63-c60c3cbde7a0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:25.242116 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66c66a-17ae-420d-af63-c60c3cbde7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:25.242309 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66c66a-17ae-420d-af63-c60c3cbde7a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:25.242925 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66c66a-17ae-420d-af63-c60c3cbde7a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:25.243270 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66c66a-17ae-420d-af63-c60c3cbde7a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:25.244687 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66c66a-17ae-420d-af63-c60c3cbde7a0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:25.261461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 527/1046] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:25.267552 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66c66a-17ae-420d-af63-c60c3cbde7a0 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:25.355381 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53511a4e-4697-42d2-9fc8-3c5bf8d93be1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:25.385412 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66c66a-17ae-420d-af63-c60c3cbde7a0 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:25.390901 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:25.398347 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1e898-ba29-4266-8f8a-7d683e87f429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:25.401348 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1e898-ba29-4266-8f8a-7d683e87f429 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:25.401348 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1e898-ba29-4266-8f8a-7d683e87f429 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:25.401348 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1e898-ba29-4266-8f8a-7d683e87f429 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:25.401348 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e1e898-ba29-4266-8f8a-7d683e87f429 None admin] RBAC: Adding query filter params (type=image) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:25.401348 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e1e898-ba29-4266-8f8a-7d683e87f429 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:25.401990 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e1e898-ba29-4266-8f8a-7d683e87f429 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:25.408410 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 520/1047] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:24:25 2022] GET /identity/v3/services?type=image => generated 350 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:25.414330 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1e898-ba29-4266-8f8a-7d683e87f429 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:25.424720 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddcab96-8570-4cb4-b465-94a8549d0597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:25.425359 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddcab96-8570-4cb4-b465-94a8549d0597 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:25.425639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddcab96-8570-4cb4-b465-94a8549d0597 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:25.425911 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddcab96-8570-4cb4-b465-94a8549d0597 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:25.432985 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddcab96-8570-4cb4-b465-94a8549d0597 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:25.434547 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddcab96-8570-4cb4-b465-94a8549d0597 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:25.449050 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-bddcab96-8570-4cb4-b465-94a8549d0597 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:24:25.449050 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:25.449050 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:25.449050 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:25.449050 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:25.449050 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:25.449050 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:25.449050 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:25.449050 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:25.449050 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:25.449050 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:25.449050 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:25.449050 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:24:25.449050 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:24:25.449050 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:24:25.449962 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:24:25.449962 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:25.449962 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:25.449962 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:25.449962 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:25.449962 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:25.449962 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:25.449962 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:25.449962 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:25.449962 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:25.449962 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:25.449962 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:25.449962 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:25.449962 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:25.449962 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:25.449962 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:25.449962 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:25.450849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:24:25.450849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:24:25.450849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:24:25.450849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:24:25.450849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:24:25.450849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:24:25.450849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:24:25.450849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:25.450849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 528/1048] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:24:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:24:25.457055 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddcab96-8570-4cb4-b465-94a8549d0597 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:25.460553 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:25.529814 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1e898-ba29-4266-8f8a-7d683e87f429 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:25.567324 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9603198b-5444-45ad-8349-414c0381c210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:25.567937 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603198b-5444-45ad-8349-414c0381c210 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:25.568072 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddcab96-8570-4cb4-b465-94a8549d0597 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:25.568244 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603198b-5444-45ad-8349-414c0381c210 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:25.568487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603198b-5444-45ad-8349-414c0381c210 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:25.569197 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9603198b-5444-45ad-8349-414c0381c210 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:25.571051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9603198b-5444-45ad-8349-414c0381c210 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:25.602707 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a4445-33ae-4802-a0b2-cc9a12162588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:25.603356 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a4445-33ae-4802-a0b2-cc9a12162588 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:25.603630 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a4445-33ae-4802-a0b2-cc9a12162588 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:25.603906 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a4445-33ae-4802-a0b2-cc9a12162588 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:25.604592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39a4445-33ae-4802-a0b2-cc9a12162588 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:25.604900 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39a4445-33ae-4802-a0b2-cc9a12162588 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:25.606322 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39a4445-33ae-4802-a0b2-cc9a12162588 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:25.618294 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 529/1049] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:24:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:25.624551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a4445-33ae-4802-a0b2-cc9a12162588 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:25.659210 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 521/1050] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:24:25 2022] POST /identity/v3/endpoints => generated 327 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:24:25.704516 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:25.749195 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a4445-33ae-4802-a0b2-cc9a12162588 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:25.793466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d61f1-cb0c-46a1-8834-20e7222c61aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:25.794086 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d61f1-cb0c-46a1-8834-20e7222c61aa None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:25.794365 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d61f1-cb0c-46a1-8834-20e7222c61aa None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:25.794635 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d61f1-cb0c-46a1-8834-20e7222c61aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:25.795384 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d61f1-cb0c-46a1-8834-20e7222c61aa None admin] RBAC: Adding query filter params (role.id=eb8debe34fab43f683a44c57759db243, scope.project.id=8e201e5f281a4977be503e22daf149d7, user.id=eb3f6deffb54467f8b923956ed2a6acf) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:25.795694 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d61f1-cb0c-46a1-8834-20e7222c61aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:25.797119 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d61f1-cb0c-46a1-8834-20e7222c61aa None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:25.804778 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 530/1051] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:24:25 2022] GET /identity/v3/role_assignments?user.id=eb3f6deffb54467f8b923956ed2a6acf&scope.project.id=8e201e5f281a4977be503e22daf149d7&role.id=eb8debe34fab43f683a44c57759db243 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:27.290760 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1359696-d200-4ef8-8c2f-730cfd8a7bcc None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:27.291327 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1359696-d200-4ef8-8c2f-730cfd8a7bcc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:27.291327 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1359696-d200-4ef8-8c2f-730cfd8a7bcc None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:27.292823 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 522/1052] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:27.297702 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d46984-ba60-4526-a0e1-9bf7408a2ff8 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:27.298105 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d46984-ba60-4526-a0e1-9bf7408a2ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:27.298281 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d46984-ba60-4526-a0e1-9bf7408a2ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:27.316547 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e8d46984-ba60-4526-a0e1-9bf7408a2ff8 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:27.317733 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e8d46984-ba60-4526-a0e1-9bf7408a2ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:27.387345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 531/1053] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:27.408813 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1359696-d200-4ef8-8c2f-730cfd8a7bcc None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:27.464334 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:27.515634 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1359696-d200-4ef8-8c2f-730cfd8a7bcc None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:27.550226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35461d19-2395-4d5d-bb79-d31a46d1ab14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:27.550773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35461d19-2395-4d5d-bb79-d31a46d1ab14 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:27.551047 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35461d19-2395-4d5d-bb79-d31a46d1ab14 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:27.551347 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35461d19-2395-4d5d-bb79-d31a46d1ab14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:27.561959 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35461d19-2395-4d5d-bb79-d31a46d1ab14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:24:27.567055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-35461d19-2395-4d5d-bb79-d31a46d1ab14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:24:27.567055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:27.567055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:27.567055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:27.567055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:27.567055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:27.567055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:27.567055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:27.567055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:27.567055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:27.567055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:27.567055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:27.567055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:24:27.567055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:24:27.567055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:24:27.567055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:24:27.568119 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:24:27.568119 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:24:27.568119 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:24:27.568119 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:24:27.568119 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:27.568119 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:27.568119 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:27.568119 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:27.568119 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:27.568119 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:27.568119 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:27.568119 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:27.568119 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:27.568119 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:27.568119 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:27.568119 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:27.569022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:27.569022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:27.569022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:27.569022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:27.569022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:24:27.569022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:24:27.569022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:24:27.569022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:24:27.569022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:24:27.569022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:24:27.569022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:24:27.569022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:27.569022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 523/1054] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:24:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:27.575971 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d46984-ba60-4526-a0e1-9bf7408a2ff8 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:27.629133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:27.681312 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d46984-ba60-4526-a0e1-9bf7408a2ff8 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:27.712106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6ffa5-4312-4132-b225-56599b878352 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:27.712401 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6ffa5-4312-4132-b225-56599b878352 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:27.712679 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6ffa5-4312-4132-b225-56599b878352 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:27.714028 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 524/1055] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:24:27.716956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a09426d-5113-4638-a021-7a45613e8d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:27.717596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09426d-5113-4638-a021-7a45613e8d6e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:27.717841 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09426d-5113-4638-a021-7a45613e8d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:27.718112 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09426d-5113-4638-a021-7a45613e8d6e None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:27.718767 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a09426d-5113-4638-a021-7a45613e8d6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:27.719068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a09426d-5113-4638-a021-7a45613e8d6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:27.719301 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd402f-e520-4ef8-87f4-14c1fdefd4fc None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:27.719684 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd402f-e520-4ef8-87f4-14c1fdefd4fc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:27.720013 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd402f-e520-4ef8-87f4-14c1fdefd4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:27.720445 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a09426d-5113-4638-a021-7a45613e8d6e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:27.731211 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 532/1056] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:24:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:27.735911 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a09426d-5113-4638-a021-7a45613e8d6e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:27.740499 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1dfd402f-e520-4ef8-87f4-14c1fdefd4fc None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:27.742317 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1dfd402f-e520-4ef8-87f4-14c1fdefd4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:27.815701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 525/1057] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:27.830311 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4fbef9-6322-4017-b9d7-2731a6f528e7 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:27.830603 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4fbef9-6322-4017-b9d7-2731a6f528e7 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:27.830880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4fbef9-6322-4017-b9d7-2731a6f528e7 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:27.832442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 526/1058] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:24:27.837100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77ce07-2484-41f7-af45-56fe69bdae62 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:27.837588 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77ce07-2484-41f7-af45-56fe69bdae62 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:27.837876 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77ce07-2484-41f7-af45-56fe69bdae62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:27.842183 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a09426d-5113-4638-a021-7a45613e8d6e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:27.855984 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6d77ce07-2484-41f7-af45-56fe69bdae62 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:27.856561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:27.857207 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6d77ce07-2484-41f7-af45-56fe69bdae62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:27.877461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45efce15-7ee3-441a-8776-2429cd4cc8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:27.878096 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efce15-7ee3-441a-8776-2429cd4cc8fc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:27.878379 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efce15-7ee3-441a-8776-2429cd4cc8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:27.878653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efce15-7ee3-441a-8776-2429cd4cc8fc None admin] PATH_INFO: `/v3/users/bb7e01843be144bbbe0939ad5f7bd79f` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:27.892121 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45efce15-7ee3-441a-8776-2429cd4cc8fc None admin] RBAC: Authorizing `identity:get_user(user_id=bb7e01843be144bbbe0939ad5f7bd79f)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:27.892819 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:27.893566 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45efce15-7ee3-441a-8776-2429cd4cc8fc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:27.895395 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 533/1059] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:24:27 2022] GET /identity/v3/users/bb7e01843be144bbbe0939ad5f7bd79f => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:27.896052 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45efce15-7ee3-441a-8776-2429cd4cc8fc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:27.927578 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 527/1060] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:24:27.928647 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77ce07-2484-41f7-af45-56fe69bdae62 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:27.997938 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45efce15-7ee3-441a-8776-2429cd4cc8fc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:28.030369 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77ce07-2484-41f7-af45-56fe69bdae62 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:28.034032 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f3ab8-e80b-4dc2-bab9-7b1b62c3d5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:28.034032 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f3ab8-e80b-4dc2-bab9-7b1b62c3d5b9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:28.034032 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f3ab8-e80b-4dc2-bab9-7b1b62c3d5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:28.034032 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f3ab8-e80b-4dc2-bab9-7b1b62c3d5b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:28.064641 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec26d02-4030-4550-b796-828277b22ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:28.065251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec26d02-4030-4550-b796-828277b22ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:28.067829 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec26d02-4030-4550-b796-828277b22ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:28.067829 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec26d02-4030-4550-b796-828277b22ef5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:28.075011 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:28.101809 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f3ab8-e80b-4dc2-bab9-7b1b62c3d5b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:28.103342 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f3ab8-e80b-4dc2-bab9-7b1b62c3d5b9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:28.109557 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:28.135862 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec26d02-4030-4550-b796-828277b22ef5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:28.137369 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec26d02-4030-4550-b796-828277b22ef5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:28.202851 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 534/1061] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:24:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:24:28.203531 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f3ab8-e80b-4dc2-bab9-7b1b62c3d5b9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:28.242450 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 528/1062] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:24:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:24:28.243311 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec26d02-4030-4550-b796-828277b22ef5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:28.280362 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:28.310689 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f3ab8-e80b-4dc2-bab9-7b1b62c3d5b9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:28.313475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:28.344049 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec26d02-4030-4550-b796-828277b22ef5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:28.345121 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8e9ac-26b8-4bad-a3cf-37e929b3c76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:28.345754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8e9ac-26b8-4bad-a3cf-37e929b3c76a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:28.346032 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8e9ac-26b8-4bad-a3cf-37e929b3c76a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:28.346302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8e9ac-26b8-4bad-a3cf-37e929b3c76a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:28.356303 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac8e9ac-26b8-4bad-a3cf-37e929b3c76a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:24:28.361034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-cac8e9ac-26b8-4bad-a3cf-37e929b3c76a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 12:24:28.361034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:28.361034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:28.361034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:28.361034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:28.361034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:28.361034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:28.361034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:28.361034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:28.361034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:28.361034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:28.361034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:28.361034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:24:28.361034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:24:28.361034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:24:28.361034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:24:28.361979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:24:28.361979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:24:28.361979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:24:28.361979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:24:28.361979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:28.361979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:28.361979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:28.361979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:28.361979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:28.361979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:28.361979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:28.361979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:28.361979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:28.361979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:28.361979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:28.361979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:28.362822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:28.362822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:28.362822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:28.362822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:28.362822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:24:28.362822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:24:28.362822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:24:28.362822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:24:28.362822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:24:28.362822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:24:28.362822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 12:24:28.362822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:28.362822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 535/1063] 10.222.0.49 () {56 vars in 1120 bytes} [Wed Aug 10 12:24:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:28.363559 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8e9ac-26b8-4bad-a3cf-37e929b3c76a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:28.383876 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87149df9-98d0-41bd-bebb-c187226813fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:28.384498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87149df9-98d0-41bd-bebb-c187226813fb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:28.384771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87149df9-98d0-41bd-bebb-c187226813fb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:28.385039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87149df9-98d0-41bd-bebb-c187226813fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:28.395546 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87149df9-98d0-41bd-bebb-c187226813fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:28.397005 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87149df9-98d0-41bd-bebb-c187226813fb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:28.411194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-87149df9-98d0-41bd-bebb-c187226813fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:28.411194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:28.411194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:28.411194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:28.411194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:28.411194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:28.411194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:28.411194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:28.411194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:28.411194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:28.411194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:28.411194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:28.411194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:24:28.411194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:24:28.411194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:24:28.411194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:24:28.412313 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:28.412313 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:28.412313 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:28.412313 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:28.412313 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:28.412313 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:28.412313 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:28.412313 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:28.412313 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:28.412313 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:28.412313 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:28.412313 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:28.412313 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:28.412313 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:28.412313 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:28.412313 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:28.412313 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:24:28.414200 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:24:28.414200 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:28.414200 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:28.414200 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 529/1064] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:24:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:28.414200 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87149df9-98d0-41bd-bebb-c187226813fb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:28.469291 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8e9ac-26b8-4bad-a3cf-37e929b3c76a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:28.498012 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:28.502896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b074a0-eea2-4a2b-a780-6af18c362aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:28.503560 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b074a0-eea2-4a2b-a780-6af18c362aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:28.503846 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b074a0-eea2-4a2b-a780-6af18c362aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:28.504120 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b074a0-eea2-4a2b-a780-6af18c362aa3 None admin] PATH_INFO: `/v3/projects/c53868d088284b638b923a07fd036ce6` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:28.505241 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b074a0-eea2-4a2b-a780-6af18c362aa3 None admin] RBAC: Authorizing `identity:get_project(project_id=c53868d088284b638b923a07fd036ce6)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:28.507274 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b074a0-eea2-4a2b-a780-6af18c362aa3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:28.510104 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 536/1065] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:24:28 2022] GET /identity/v3/projects/c53868d088284b638b923a07fd036ce6 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:28.510104 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b074a0-eea2-4a2b-a780-6af18c362aa3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:28.516134 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87149df9-98d0-41bd-bebb-c187226813fb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:28.539891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:28.550979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe6ea6-b6c8-43fc-a2e8-e86b8c338ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:28.551756 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe6ea6-b6c8-43fc-a2e8-e86b8c338ffc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:28.552037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe6ea6-b6c8-43fc-a2e8-e86b8c338ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:28.552320 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe6ea6-b6c8-43fc-a2e8-e86b8c338ffc None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:28.552974 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe6ea6-b6c8-43fc-a2e8-e86b8c338ffc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:28.553287 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe6ea6-b6c8-43fc-a2e8-e86b8c338ffc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:28.554679 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe6ea6-b6c8-43fc-a2e8-e86b8c338ffc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:28.563122 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 530/1066] 10.222.0.49 () {56 vars in 1115 bytes} [Wed Aug 10 12:24:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:28.563939 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe6ea6-b6c8-43fc-a2e8-e86b8c338ffc None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:28.614114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b074a0-eea2-4a2b-a780-6af18c362aa3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:28.648597 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d7eb8-0e89-4a5b-9a4e-2ed386e8dea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:28.649203 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d7eb8-0e89-4a5b-9a4e-2ed386e8dea8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:28.649484 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d7eb8-0e89-4a5b-9a4e-2ed386e8dea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:28.649756 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d7eb8-0e89-4a5b-9a4e-2ed386e8dea8 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:28.650389 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d7eb8-0e89-4a5b-9a4e-2ed386e8dea8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:28.650694 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d7eb8-0e89-4a5b-9a4e-2ed386e8dea8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:28.652091 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d7eb8-0e89-4a5b-9a4e-2ed386e8dea8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:28.662983 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe6ea6-b6c8-43fc-a2e8-e86b8c338ffc None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:28.663521 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 537/1067] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:28.664315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d7eb8-0e89-4a5b-9a4e-2ed386e8dea8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:28.701371 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79de20-4794-42ec-a971-980f7baffdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:28.701992 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79de20-4794-42ec-a971-980f7baffdcd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:28.702275 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79de20-4794-42ec-a971-980f7baffdcd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:28.702548 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79de20-4794-42ec-a971-980f7baffdcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:28.703326 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de79de20-4794-42ec-a971-980f7baffdcd None admin] RBAC: Adding query filter params (role.id=bffbe227cda1418cbb58394ddff6f861, scope.project.id=c53868d088284b638b923a07fd036ce6, user.id=bb7e01843be144bbbe0939ad5f7bd79f) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:28.703611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de79de20-4794-42ec-a971-980f7baffdcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:28.705053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de79de20-4794-42ec-a971-980f7baffdcd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:28.712445 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 531/1068] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:24:28 2022] GET /identity/v3/role_assignments?user.id=bb7e01843be144bbbe0939ad5f7bd79f&scope.project.id=c53868d088284b638b923a07fd036ce6&role.id=bffbe227cda1418cbb58394ddff6f861 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:28.746462 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:28.795501 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d7eb8-0e89-4a5b-9a4e-2ed386e8dea8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:28.837769 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a512e-c190-4218-bde6-8caa15fb362a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:28.838426 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a512e-c190-4218-bde6-8caa15fb362a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:28.838705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a512e-c190-4218-bde6-8caa15fb362a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:28.838979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a512e-c190-4218-bde6-8caa15fb362a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:28.851131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a512e-c190-4218-bde6-8caa15fb362a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:28.852704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a512e-c190-4218-bde6-8caa15fb362a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:28.870188 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-354a512e-c190-4218-bde6-8caa15fb362a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:28.870188 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:28.870188 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:28.870188 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:28.870188 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:28.870188 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:28.870188 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:28.870188 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:28.870188 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:28.870188 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:28.870188 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:28.870188 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:28.870188 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:24:28.870188 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:24:28.870188 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:24:28.870188 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:24:28.871158 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:28.871158 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:28.871158 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:28.871158 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:28.871158 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:28.871158 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:28.871158 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:28.871158 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:28.871158 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:28.871158 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:28.871158 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:28.871158 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:28.871158 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:28.871158 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:28.871158 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:28.871158 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:28.871158 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:24:28.872378 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:24:28.872378 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:28.872378 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:28.873259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 538/1069] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:24:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:28.879328 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79de20-4794-42ec-a971-980f7baffdcd None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:28.935920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:28.989037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79de20-4794-42ec-a971-980f7baffdcd None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:29.023565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95259a-af05-48e3-bdde-46c1db7692e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:29.024176 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95259a-af05-48e3-bdde-46c1db7692e5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:29.024457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95259a-af05-48e3-bdde-46c1db7692e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:29.024729 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95259a-af05-48e3-bdde-46c1db7692e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:29.025366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c95259a-af05-48e3-bdde-46c1db7692e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:29.025676 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c95259a-af05-48e3-bdde-46c1db7692e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:29.027047 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c95259a-af05-48e3-bdde-46c1db7692e5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:29.038923 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 532/1070] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:29.044672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a512e-c190-4218-bde6-8caa15fb362a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:29.143541 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a512e-c190-4218-bde6-8caa15fb362a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:29.156326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:29.176932 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5162b5-86a4-4dac-8bd7-1224695eddd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:29.177635 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5162b5-86a4-4dac-8bd7-1224695eddd7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:29.177917 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5162b5-86a4-4dac-8bd7-1224695eddd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:29.180235 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5162b5-86a4-4dac-8bd7-1224695eddd7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:29.186355 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5162b5-86a4-4dac-8bd7-1224695eddd7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:29.187811 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5162b5-86a4-4dac-8bd7-1224695eddd7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:29.199965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-9a5162b5-86a4-4dac-8bd7-1224695eddd7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:24:29.199965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:29.199965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:29.199965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:29.199965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:29.199965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:29.199965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:29.199965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:29.199965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:29.199965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:29.199965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:29.199965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:29.199965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:24:29.199965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:24:29.199965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:24:29.199965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:24:29.201002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:29.201002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:29.201002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:24:29.201002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:24:29.201002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:24:29.201002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:24:29.201002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:29.201002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:29.201002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:29.201002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:29.201002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:29.201002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:29.201002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:29.201002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:29.201002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:29.201002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:29.201002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:29.201002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:29.202283 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:29.202283 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:29.202283 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:24:29.202283 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:24:29.202283 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:24:29.202283 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:24:29.202283 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:24:29.202283 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:24:29.202283 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:24:29.202283 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:29.202283 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 539/1071] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Aug 10 12:24:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:29.206770 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95259a-af05-48e3-bdde-46c1db7692e5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:29.307386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95259a-af05-48e3-bdde-46c1db7692e5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:29.317643 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:29.340431 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f433e2-11cf-4a05-89d4-3548a400b6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:29.341031 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f433e2-11cf-4a05-89d4-3548a400b6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:29.341306 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f433e2-11cf-4a05-89d4-3548a400b6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:29.341578 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f433e2-11cf-4a05-89d4-3548a400b6d7 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:29.342289 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f433e2-11cf-4a05-89d4-3548a400b6d7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:29.342592 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f433e2-11cf-4a05-89d4-3548a400b6d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:29.344046 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f433e2-11cf-4a05-89d4-3548a400b6d7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:29.362396 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 533/1072] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Aug 10 12:24:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:29.368535 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5162b5-86a4-4dac-8bd7-1224695eddd7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:29.464679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5162b5-86a4-4dac-8bd7-1224695eddd7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:29.498734 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46cecc-f4c7-4be0-88d1-9571f440fe05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:29.499388 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46cecc-f4c7-4be0-88d1-9571f440fe05 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:29.499668 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46cecc-f4c7-4be0-88d1-9571f440fe05 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:29.499938 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46cecc-f4c7-4be0-88d1-9571f440fe05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:29.510249 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b46cecc-f4c7-4be0-88d1-9571f440fe05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:29.511736 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b46cecc-f4c7-4be0-88d1-9571f440fe05 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:29.525779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-6b46cecc-f4c7-4be0-88d1-9571f440fe05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:29.525779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:29.525779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:29.525779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:29.525779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:29.525779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:29.525779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:29.525779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:29.525779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:29.525779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:29.525779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:29.525779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:29.525779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:24:29.525779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:24:29.525779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:24:29.525779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:24:29.526695 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:29.526695 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:29.526695 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:29.526695 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:29.526695 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:29.526695 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:29.526695 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:29.526695 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:29.526695 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:29.526695 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:29.526695 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:29.526695 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:29.526695 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:29.526695 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:29.526695 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:29.526695 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:29.526695 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:24:29.527635 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:24:29.527635 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:29.527635 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:29.527635 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 540/1073] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:24:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:29.532689 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f433e2-11cf-4a05-89d4-3548a400b6d7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:29.629815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f433e2-11cf-4a05-89d4-3548a400b6d7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:29.662289 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad469834-d167-47d4-9571-dadb648bf092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:29.662876 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad469834-d167-47d4-9571-dadb648bf092 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:29.663348 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad469834-d167-47d4-9571-dadb648bf092 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:29.663625 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad469834-d167-47d4-9571-dadb648bf092 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:29.664250 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad469834-d167-47d4-9571-dadb648bf092 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:29.664549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad469834-d167-47d4-9571-dadb648bf092 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:29.665914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad469834-d167-47d4-9571-dadb648bf092 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:29.676713 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 534/1074] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:29.682913 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46cecc-f4c7-4be0-88d1-9571f440fe05 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:29.709013 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:29.782086 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46cecc-f4c7-4be0-88d1-9571f440fe05 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:29.816295 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8dbef-5ad4-43a5-a89d-e72b43188c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:29.816953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8dbef-5ad4-43a5-a89d-e72b43188c5e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:29.817234 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8dbef-5ad4-43a5-a89d-e72b43188c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:29.817510 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8dbef-5ad4-43a5-a89d-e72b43188c5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:29.825986 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8dbef-5ad4-43a5-a89d-e72b43188c5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:29.827479 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8dbef-5ad4-43a5-a89d-e72b43188c5e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:29.839231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-4ff8dbef-5ad4-43a5-a89d-e72b43188c5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:24:29.839231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:29.839231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:29.839231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:29.839231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:29.839231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:29.839231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:29.839231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:29.839231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:29.839231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:29.839231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:29.839231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:29.839231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:24:29.839231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:24:29.839231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:24:29.840305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:24:29.840305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:29.840305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:29.840305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:29.840305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:29.840305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:29.840305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:29.840305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:29.840305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:29.840305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:29.840305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:29.840305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:29.840305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:29.840305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:29.840305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:29.840305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:29.840305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:29.841381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:24:29.841381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:24:29.841381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:24:29.841381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:24:29.841381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:24:29.841381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:24:29.841381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:24:29.841381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:29.843328 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 541/1075] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:24:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:29.846030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad469834-d167-47d4-9571-dadb648bf092 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:29.875350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:29.945883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad469834-d167-47d4-9571-dadb648bf092 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:29.979033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7bc3d6-a2db-4844-8411-34318640f37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:29.979682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7bc3d6-a2db-4844-8411-34318640f37a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:29.979965 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7bc3d6-a2db-4844-8411-34318640f37a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:29.980243 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7bc3d6-a2db-4844-8411-34318640f37a None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:29.980926 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7bc3d6-a2db-4844-8411-34318640f37a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:29.981233 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7bc3d6-a2db-4844-8411-34318640f37a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:29.982658 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7bc3d6-a2db-4844-8411-34318640f37a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:29.994773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 535/1076] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:24:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:30.001245 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8dbef-5ad4-43a5-a89d-e72b43188c5e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:30.082656 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:30.098300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8dbef-5ad4-43a5-a89d-e72b43188c5e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:30.133160 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70001cef-3ca0-499c-98fa-4943c5f348e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:30.133811 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70001cef-3ca0-499c-98fa-4943c5f348e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:30.134092 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70001cef-3ca0-499c-98fa-4943c5f348e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:30.134370 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70001cef-3ca0-499c-98fa-4943c5f348e2 None admin] PATH_INFO: `/v3/projects/8e201e5f281a4977be503e22daf149d7/users/eb3f6deffb54467f8b923956ed2a6acf/roles/eb8debe34fab43f683a44c57759db243` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:30.136555 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70001cef-3ca0-499c-98fa-4943c5f348e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e201e5f281a4977be503e22daf149d7, user_id=eb3f6deffb54467f8b923956ed2a6acf, role_id=eb8debe34fab43f683a44c57759db243)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:30.137984 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70001cef-3ca0-499c-98fa-4943c5f348e2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:30.248533 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 542/1077] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 12:24:30 2022] PUT /identity/v3/projects/8e201e5f281a4977be503e22daf149d7/users/eb3f6deffb54467f8b923956ed2a6acf/roles/eb8debe34fab43f683a44c57759db243 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:24:30.721385 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d0971-6d71-478c-b581-9524b7a56ce4 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:30.721914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d0971-6d71-478c-b581-9524b7a56ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:30.721914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d0971-6d71-478c-b581-9524b7a56ce4 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:30.723286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 536/1078] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:30.728990 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21ea52-537c-4941-a15c-9f94f79a5700 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:30.729388 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21ea52-537c-4941-a15c-9f94f79a5700 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:30.729668 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21ea52-537c-4941-a15c-9f94f79a5700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:30.748380 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6e21ea52-537c-4941-a15c-9f94f79a5700 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:30.749582 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6e21ea52-537c-4941-a15c-9f94f79a5700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:30.814959 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 543/1079] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:30.836989 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d0971-6d71-478c-b581-9524b7a56ce4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:30.925793 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:30.938252 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d0971-6d71-478c-b581-9524b7a56ce4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:30.973205 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b0ae8-deca-492a-92ac-28e981e715e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:30.973768 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b0ae8-deca-492a-92ac-28e981e715e0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:30.974058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b0ae8-deca-492a-92ac-28e981e715e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:30.974332 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b0ae8-deca-492a-92ac-28e981e715e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:30.984576 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b0ae8-deca-492a-92ac-28e981e715e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:24:30.989236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-3e7b0ae8-deca-492a-92ac-28e981e715e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:24:30.989236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:30.989236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:30.989236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:30.989236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:30.989236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:30.989236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:30.989236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:30.989236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:30.989236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:30.989236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:30.989236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:30.989236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:24:30.989236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:24:30.989236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:24:30.989236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:24:30.990354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:24:30.990354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:24:30.990354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:24:30.990354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:24:30.990354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:30.990354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:30.990354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:30.990354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:30.990354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:30.990354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:30.990354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:30.990354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:30.990354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:30.990354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:30.990354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:30.990354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:30.991564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:30.991564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:30.991564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:30.991564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:30.991564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:24:30.991564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:24:30.991564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:24:30.991564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:24:30.991564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:24:30.991564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:24:30.991564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:24:30.991564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:30.995337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 537/1080] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:24:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:30.997027 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e21ea52-537c-4941-a15c-9f94f79a5700 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:31.070864 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:31.094487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e21ea52-537c-4941-a15c-9f94f79a5700 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:31.129677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a10f6-7f32-474e-99a7-cf338f3e5658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:31.130313 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a10f6-7f32-474e-99a7-cf338f3e5658 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:31.130597 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a10f6-7f32-474e-99a7-cf338f3e5658 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:31.130873 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a10f6-7f32-474e-99a7-cf338f3e5658 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:31.131580 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a10f6-7f32-474e-99a7-cf338f3e5658 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:31.132490 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a10f6-7f32-474e-99a7-cf338f3e5658 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:31.133849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a10f6-7f32-474e-99a7-cf338f3e5658 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:31.141794 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 544/1081] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:24:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:31.148069 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b0ae8-deca-492a-92ac-28e981e715e0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:31.245659 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b0ae8-deca-492a-92ac-28e981e715e0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:31.282991 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f4997e-345f-4c16-b340-a80f8dcb8409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:31.283658 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f4997e-345f-4c16-b340-a80f8dcb8409 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:31.283938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f4997e-345f-4c16-b340-a80f8dcb8409 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:31.284209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f4997e-345f-4c16-b340-a80f8dcb8409 None admin] PATH_INFO: `/v3/users/bb7e01843be144bbbe0939ad5f7bd79f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:31.285321 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f4997e-345f-4c16-b340-a80f8dcb8409 None admin] RBAC: Authorizing `identity:get_user(user_id=bb7e01843be144bbbe0939ad5f7bd79f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:31.286758 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f4997e-345f-4c16-b340-a80f8dcb8409 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:31.288593 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 538/1082] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:24:31 2022] GET /identity/v3/users/bb7e01843be144bbbe0939ad5f7bd79f => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:31.294238 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a10f6-7f32-474e-99a7-cf338f3e5658 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:31.393750 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a10f6-7f32-474e-99a7-cf338f3e5658 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:31.426375 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc959bb-dee3-4550-aa49-eae9b47a35e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:31.427004 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc959bb-dee3-4550-aa49-eae9b47a35e1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:31.427327 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc959bb-dee3-4550-aa49-eae9b47a35e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:31.427603 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc959bb-dee3-4550-aa49-eae9b47a35e1 None admin] PATH_INFO: `/v3/projects/c53868d088284b638b923a07fd036ce6` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:31.429268 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc959bb-dee3-4550-aa49-eae9b47a35e1 None admin] RBAC: Authorizing `identity:get_project(project_id=c53868d088284b638b923a07fd036ce6)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:31.430778 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc959bb-dee3-4550-aa49-eae9b47a35e1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:31.432452 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 545/1083] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:24:31 2022] GET /identity/v3/projects/c53868d088284b638b923a07fd036ce6 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:31.438330 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f4997e-345f-4c16-b340-a80f8dcb8409 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:31.457936 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:31.537806 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f4997e-345f-4c16-b340-a80f8dcb8409 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:31.570780 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c967903c-ebe5-480c-a029-6dd9347345b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:31.571456 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967903c-ebe5-480c-a029-6dd9347345b2 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:31.571734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967903c-ebe5-480c-a029-6dd9347345b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:31.572001 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967903c-ebe5-480c-a029-6dd9347345b2 None admin] PATH_INFO: `/v3/projects/c53868d088284b638b923a07fd036ce6/users/bb7e01843be144bbbe0939ad5f7bd79f/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:31.573858 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c967903c-ebe5-480c-a029-6dd9347345b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=c53868d088284b638b923a07fd036ce6, user_id=bb7e01843be144bbbe0939ad5f7bd79f, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:31.575303 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c967903c-ebe5-480c-a029-6dd9347345b2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:31.620775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 539/1084] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 12:24:31 2022] PUT /identity/v3/projects/c53868d088284b638b923a07fd036ce6/users/bb7e01843be144bbbe0939ad5f7bd79f/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:24:32.238583 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ea74d-fc3c-489f-ad38-6536e27bca6e None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:32.239113 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ea74d-fc3c-489f-ad38-6536e27bca6e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:32.239113 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ea74d-fc3c-489f-ad38-6536e27bca6e None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:32.240507 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 546/1085] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:32.246393 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24697c-0fd0-445d-b479-bd06f3ef3fdd None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:32.247085 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24697c-0fd0-445d-b479-bd06f3ef3fdd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:32.247216 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24697c-0fd0-445d-b479-bd06f3ef3fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:32.265737 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9d24697c-0fd0-445d-b479-bd06f3ef3fdd None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:32.266943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9d24697c-0fd0-445d-b479-bd06f3ef3fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:32.333744 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 540/1086] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:32.357282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ea74d-fc3c-489f-ad38-6536e27bca6e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:32.361080 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:32.460339 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ea74d-fc3c-489f-ad38-6536e27bca6e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:32.493661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c5ba2-8284-4df8-8c7a-d4da2c17713e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:32.494225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c5ba2-8284-4df8-8c7a-d4da2c17713e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:32.494492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c5ba2-8284-4df8-8c7a-d4da2c17713e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:32.494760 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c5ba2-8284-4df8-8c7a-d4da2c17713e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:32.504760 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c5ba2-8284-4df8-8c7a-d4da2c17713e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:24:32.509410 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-3d7c5ba2-8284-4df8-8c7a-d4da2c17713e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 12:24:32.509410 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:32.509410 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:32.509410 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:32.509410 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:32.509410 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:32.509410 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:32.509410 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:32.509410 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:32.509410 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:32.509410 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:32.509410 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:32.509410 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:24:32.509410 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:24:32.509410 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:24:32.509410 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:24:32.510438 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:24:32.510438 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:24:32.510438 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:24:32.510438 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:24:32.510438 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:32.510438 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:32.510438 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:32.510438 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:32.510438 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:32.510438 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:32.510438 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:32.510438 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:32.510438 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:32.510438 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:32.510438 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:32.510438 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:32.511324 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:32.511324 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:32.511324 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:32.511324 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:32.511324 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:24:32.511324 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:24:32.511324 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:24:32.511324 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:24:32.511324 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:24:32.511324 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:24:32.511324 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 12:24:32.511324 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:32.511324 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 547/1087] 10.222.0.49 () {56 vars in 1120 bytes} [Wed Aug 10 12:24:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:32.516585 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24697c-0fd0-445d-b479-bd06f3ef3fdd None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:32.524824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:32.614472 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24697c-0fd0-445d-b479-bd06f3ef3fdd None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:32.648102 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd017944-b516-4916-b49e-2e6b5b579b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:32.648742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd017944-b516-4916-b49e-2e6b5b579b7d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:32.649023 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd017944-b516-4916-b49e-2e6b5b579b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:32.649297 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd017944-b516-4916-b49e-2e6b5b579b7d None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:32.649951 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd017944-b516-4916-b49e-2e6b5b579b7d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:32.650255 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd017944-b516-4916-b49e-2e6b5b579b7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:32.651635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd017944-b516-4916-b49e-2e6b5b579b7d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:32.659584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 541/1088] 10.222.0.49 () {56 vars in 1115 bytes} [Wed Aug 10 12:24:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:32.666001 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c5ba2-8284-4df8-8c7a-d4da2c17713e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:32.729397 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:32.765963 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c5ba2-8284-4df8-8c7a-d4da2c17713e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:32.799572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f5114-822d-414d-9ae1-4a98b38ffcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:32.800201 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f5114-822d-414d-9ae1-4a98b38ffcb6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:32.800480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f5114-822d-414d-9ae1-4a98b38ffcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:32.800756 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f5114-822d-414d-9ae1-4a98b38ffcb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:32.811245 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f5114-822d-414d-9ae1-4a98b38ffcb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:32.812750 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f5114-822d-414d-9ae1-4a98b38ffcb6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:32.827461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-544f5114-822d-414d-9ae1-4a98b38ffcb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:32.827461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:32.827461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:32.827461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:32.827461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:32.827461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:32.827461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:32.827461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:32.827461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:32.827461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:32.827461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:32.827461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:32.827461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:24:32.827461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:24:32.827461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:24:32.827461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:24:32.833217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:32.833217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:32.833217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:32.833217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:32.833217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:32.833217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:32.833217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:32.833217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:32.833217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:32.833217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:32.833217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:32.833217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:32.833217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:32.833217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:32.833217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:32.833217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:32.833217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:24:32.834390 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:24:32.834390 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:32.834390 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:32.834390 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 548/1089] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:24:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:32.835646 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd017944-b516-4916-b49e-2e6b5b579b7d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:32.899977 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:32.932372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd017944-b516-4916-b49e-2e6b5b579b7d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:32.965165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc759ca4-9468-401b-91b2-9b2f65c070f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:32.965759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc759ca4-9468-401b-91b2-9b2f65c070f8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:32.966033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc759ca4-9468-401b-91b2-9b2f65c070f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:32.966301 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc759ca4-9468-401b-91b2-9b2f65c070f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:32.966927 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc759ca4-9468-401b-91b2-9b2f65c070f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:32.967250 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc759ca4-9468-401b-91b2-9b2f65c070f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:32.968591 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc759ca4-9468-401b-91b2-9b2f65c070f8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:32.980654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 542/1090] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:32.986248 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f5114-822d-414d-9ae1-4a98b38ffcb6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:33.086433 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f5114-822d-414d-9ae1-4a98b38ffcb6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:33.101808 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:33.118849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334a153-798c-432f-b321-984d2e18b26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:33.119496 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334a153-798c-432f-b321-984d2e18b26e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:33.119774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334a153-798c-432f-b321-984d2e18b26e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:33.120047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334a153-798c-432f-b321-984d2e18b26e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:33.127642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334a153-798c-432f-b321-984d2e18b26e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:33.129060 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334a153-798c-432f-b321-984d2e18b26e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:33.140818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-c334a153-798c-432f-b321-984d2e18b26e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:24:33.140818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:33.140818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:33.140818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:33.140818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:33.140818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:33.140818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:33.140818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:33.140818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:33.140818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:33.140818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:33.140818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:33.140818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:24:33.140818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:24:33.140818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:24:33.140818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:24:33.141774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:33.141774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:33.141774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:24:33.141774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:24:33.141774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:24:33.141774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:24:33.141774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:33.141774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:33.141774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:33.141774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:33.141774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:33.141774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:33.141774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:33.141774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:33.141774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:33.141774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:33.141774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:33.141774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:33.142714 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:33.142714 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:33.142714 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:24:33.142714 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:24:33.142714 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:24:33.142714 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:24:33.142714 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:24:33.142714 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:24:33.142714 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:24:33.142714 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:33.142714 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 549/1091] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Aug 10 12:24:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:33.148342 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc759ca4-9468-401b-91b2-9b2f65c070f8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:33.244601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc759ca4-9468-401b-91b2-9b2f65c070f8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:33.271161 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:33.279855 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2199ad-efa9-4371-8334-3c17b8c1e9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:33.280474 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2199ad-efa9-4371-8334-3c17b8c1e9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:33.280757 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2199ad-efa9-4371-8334-3c17b8c1e9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:33.281031 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2199ad-efa9-4371-8334-3c17b8c1e9e2 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:33.281760 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2199ad-efa9-4371-8334-3c17b8c1e9e2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:33.282069 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2199ad-efa9-4371-8334-3c17b8c1e9e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:33.283526 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2199ad-efa9-4371-8334-3c17b8c1e9e2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:33.302334 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 543/1092] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Aug 10 12:24:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:33.308785 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334a153-798c-432f-b321-984d2e18b26e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:33.406394 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334a153-798c-432f-b321-984d2e18b26e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:33.438869 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01bc060-8a46-4369-9e8f-d568d79791c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:33.439507 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bc060-8a46-4369-9e8f-d568d79791c3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:33.439790 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bc060-8a46-4369-9e8f-d568d79791c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:33.440061 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bc060-8a46-4369-9e8f-d568d79791c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:33.484722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:33.508456 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01bc060-8a46-4369-9e8f-d568d79791c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:33.509892 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01bc060-8a46-4369-9e8f-d568d79791c3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:33.604444 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 550/1093] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:24:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:24:33.604918 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bfd7f-6a6d-4283-a84c-180c835bee1c None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:33.605212 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bfd7f-6a6d-4283-a84c-180c835bee1c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:33.605512 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bfd7f-6a6d-4283-a84c-180c835bee1c None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:33.606830 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 544/1094] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:24:33.610077 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01bc060-8a46-4369-9e8f-d568d79791c3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:33.611641 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48855656-f354-4dfb-b1cc-29df5fabba4d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:33.611926 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48855656-f354-4dfb-b1cc-29df5fabba4d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:33.612198 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48855656-f354-4dfb-b1cc-29df5fabba4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:33.630379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-48855656-f354-4dfb-b1cc-29df5fabba4d None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:33.631629 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-48855656-f354-4dfb-b1cc-29df5fabba4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:33.687048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:33.698350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 545/1095] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:24:33.708948 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01bc060-8a46-4369-9e8f-d568d79791c3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:33.721637 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48855656-f354-4dfb-b1cc-29df5fabba4d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:33.743970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a46498-6802-450a-89cc-4f4bf1593d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:33.744599 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a46498-6802-450a-89cc-4f4bf1593d03 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:33.744881 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a46498-6802-450a-89cc-4f4bf1593d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:33.745157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a46498-6802-450a-89cc-4f4bf1593d03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:33.755270 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a46498-6802-450a-89cc-4f4bf1593d03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:33.756706 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a46498-6802-450a-89cc-4f4bf1593d03 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:33.771051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-86a46498-6802-450a-89cc-4f4bf1593d03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:33.771051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:33.771051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:33.771051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:33.771051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:33.771051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:33.771051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:33.771051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:33.771051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:33.771051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:33.771051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:33.771051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:33.771051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:24:33.771051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:24:33.771051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:24:33.771051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:24:33.772133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:33.772133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:33.772133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:33.772133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:33.772133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:33.772133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:33.772133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:33.772133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:33.772133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:33.772133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:33.772133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:33.772133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:33.772133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:33.772133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:33.772133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:33.772133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:33.772133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:24:33.773198 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:24:33.773198 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:24:33.773198 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:33.773198 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 551/1096] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:33.777864 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a46498-6802-450a-89cc-4f4bf1593d03 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:33.824384 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48855656-f354-4dfb-b1cc-29df5fabba4d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:33.840870 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:33.858581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fcfb22-c6e8-4cf1-bc1f-30d191b34686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:33.859231 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fcfb22-c6e8-4cf1-bc1f-30d191b34686 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:33.859516 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fcfb22-c6e8-4cf1-bc1f-30d191b34686 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:33.859796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fcfb22-c6e8-4cf1-bc1f-30d191b34686 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:33.871163 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fcfb22-c6e8-4cf1-bc1f-30d191b34686 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:24:33.876060 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-c9fcfb22-c6e8-4cf1-bc1f-30d191b34686 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:24:33.876060 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:33.876060 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:33.876060 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:33.876060 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:33.876060 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:33.876060 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:33.876060 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:33.876060 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:33.876060 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:33.876060 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:33.876060 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:33.876060 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:24:33.876060 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:24:33.876060 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:24:33.876060 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:24:33.877176 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:24:33.877176 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:24:33.877176 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:24:33.877176 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:24:33.877176 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:33.877176 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:33.877176 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:33.877176 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:33.877176 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:33.877176 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:33.877176 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:33.877176 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:33.877176 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:33.877176 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:33.877176 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:33.877176 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:33.878461 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:33.878461 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:33.878461 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:33.878461 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:33.878461 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:24:33.878461 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:24:33.878461 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:24:33.878461 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:24:33.878461 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:24:33.878461 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:24:33.878461 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:24:33.878461 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:33.878461 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 546/1097] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:24:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:24:33.879206 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a46498-6802-450a-89cc-4f4bf1593d03 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:33.883471 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fcfb22-c6e8-4cf1-bc1f-30d191b34686 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:33.904193 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:33.911812 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1bdf1-c810-4bd6-84d9-f4ff2d96fd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:33.915102 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1bdf1-c810-4bd6-84d9-f4ff2d96fd8e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:33.915102 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1bdf1-c810-4bd6-84d9-f4ff2d96fd8e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:33.915102 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1bdf1-c810-4bd6-84d9-f4ff2d96fd8e None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:33.915102 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1bdf1-c810-4bd6-84d9-f4ff2d96fd8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:33.915102 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1bdf1-c810-4bd6-84d9-f4ff2d96fd8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:33.915869 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1bdf1-c810-4bd6-84d9-f4ff2d96fd8e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:33.927447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 552/1098] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:33.934178 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1bdf1-c810-4bd6-84d9-f4ff2d96fd8e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:33.982121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fcfb22-c6e8-4cf1-bc1f-30d191b34686 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:34.016497 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0474a-08e4-4157-859e-a6f80c783144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:34.017104 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0474a-08e4-4157-859e-a6f80c783144 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:34.017382 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0474a-08e4-4157-859e-a6f80c783144 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:34.017655 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0474a-08e4-4157-859e-a6f80c783144 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:34.018305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0474a-08e4-4157-859e-a6f80c783144 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:34.018607 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0474a-08e4-4157-859e-a6f80c783144 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:34.020004 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0474a-08e4-4157-859e-a6f80c783144 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:34.028845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 547/1099] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:24:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:34.032303 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1bdf1-c810-4bd6-84d9-f4ff2d96fd8e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:34.034665 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0474a-08e4-4157-859e-a6f80c783144 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:34.067225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af63ea-b298-40a4-8dac-386ba03c315a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:34.067853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af63ea-b298-40a4-8dac-386ba03c315a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:34.068130 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af63ea-b298-40a4-8dac-386ba03c315a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:34.068399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af63ea-b298-40a4-8dac-386ba03c315a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:34.074504 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:34.075912 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af63ea-b298-40a4-8dac-386ba03c315a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:34.077402 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af63ea-b298-40a4-8dac-386ba03c315a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:34.089029 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-28af63ea-b298-40a4-8dac-386ba03c315a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:24:34.089029 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:34.089029 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:34.089029 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:34.089029 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:34.089029 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:34.089029 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:34.089029 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:34.089029 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:34.089029 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:34.089029 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:34.089029 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:34.089029 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:24:34.089029 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:24:34.089029 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:24:34.093223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:24:34.093223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:34.093223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:34.093223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:34.093223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:34.093223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:34.093223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:34.093223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:34.093223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:34.093223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:34.093223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:34.093223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:34.093223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:34.093223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:34.093223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:34.093223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:34.093223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:34.094153 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:24:34.094153 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:24:34.094153 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:24:34.094153 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:24:34.094153 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:24:34.094153 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:24:34.094153 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:24:34.094153 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:34.094153 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 553/1100] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:24:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:24:34.095750 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af63ea-b298-40a4-8dac-386ba03c315a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:34.136973 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0474a-08e4-4157-859e-a6f80c783144 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:34.144751 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:34.171593 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ae922-fce4-42ca-92be-4140774c0911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:34.172218 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ae922-fce4-42ca-92be-4140774c0911 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:34.172560 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ae922-fce4-42ca-92be-4140774c0911 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:34.172835 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ae922-fce4-42ca-92be-4140774c0911 None admin] PATH_INFO: `/v3/users/bb7e01843be144bbbe0939ad5f7bd79f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:34.173960 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ae922-fce4-42ca-92be-4140774c0911 None admin] RBAC: Authorizing `identity:get_user(user_id=bb7e01843be144bbbe0939ad5f7bd79f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:34.175436 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ae922-fce4-42ca-92be-4140774c0911 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:34.177030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 548/1101] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:24:34 2022] GET /identity/v3/users/bb7e01843be144bbbe0939ad5f7bd79f => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:34.182809 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ae922-fce4-42ca-92be-4140774c0911 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:34.197245 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af63ea-b298-40a4-8dac-386ba03c315a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:34.230742 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a11ab7-b794-4842-b900-f02caa18d322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:34.231371 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a11ab7-b794-4842-b900-f02caa18d322 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:34.231647 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a11ab7-b794-4842-b900-f02caa18d322 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:34.231916 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a11ab7-b794-4842-b900-f02caa18d322 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:34.232643 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a11ab7-b794-4842-b900-f02caa18d322 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:34.232895 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a11ab7-b794-4842-b900-f02caa18d322 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:34.234310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a11ab7-b794-4842-b900-f02caa18d322 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:34.246056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 554/1102] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:24:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:34.252190 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a11ab7-b794-4842-b900-f02caa18d322 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:34.282856 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ae922-fce4-42ca-92be-4140774c0911 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:34.290568 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:34.317943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04ee81-c832-4848-be77-9a5df83fb34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:34.318752 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04ee81-c832-4848-be77-9a5df83fb34e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:34.320046 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04ee81-c832-4848-be77-9a5df83fb34e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:34.320548 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04ee81-c832-4848-be77-9a5df83fb34e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:34.352030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a11ab7-b794-4842-b900-f02caa18d322 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:34.361572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:34.386883 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db35a245-fd3a-4070-a28c-09d10179485f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:34.387592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35a245-fd3a-4070-a28c-09d10179485f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:34.387870 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35a245-fd3a-4070-a28c-09d10179485f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:34.388139 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35a245-fd3a-4070-a28c-09d10179485f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:34.388845 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db35a245-fd3a-4070-a28c-09d10179485f None admin] RBAC: Adding query filter params (role.id=eb8debe34fab43f683a44c57759db243, scope.project.id=8e201e5f281a4977be503e22daf149d7, user.id=eb3f6deffb54467f8b923956ed2a6acf) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:34.388957 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c04ee81-c832-4848-be77-9a5df83fb34e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:34.389151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db35a245-fd3a-4070-a28c-09d10179485f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:34.390363 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c04ee81-c832-4848-be77-9a5df83fb34e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:34.390576 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db35a245-fd3a-4070-a28c-09d10179485f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:34.396671 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 555/1103] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:24:34 2022] GET /identity/v3/role_assignments?user.id=eb3f6deffb54467f8b923956ed2a6acf&scope.project.id=8e201e5f281a4977be503e22daf149d7&role.id=eb8debe34fab43f683a44c57759db243 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:34.496181 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 549/1104] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:24:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 12:24:34.502403 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db35a245-fd3a-4070-a28c-09d10179485f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:34.606066 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db35a245-fd3a-4070-a28c-09d10179485f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:34.640250 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15269a7d-0420-4c75-869a-6fbb821c4113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:34.640882 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15269a7d-0420-4c75-869a-6fbb821c4113 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:34.641155 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15269a7d-0420-4c75-869a-6fbb821c4113 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:34.641428 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15269a7d-0420-4c75-869a-6fbb821c4113 None admin] PATH_INFO: `/v3/projects/c53868d088284b638b923a07fd036ce6` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:34.642527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15269a7d-0420-4c75-869a-6fbb821c4113 None admin] RBAC: Authorizing `identity:get_project(project_id=c53868d088284b638b923a07fd036ce6)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:34.643963 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15269a7d-0420-4c75-869a-6fbb821c4113 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:34.645586 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 556/1105] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:24:34 2022] GET /identity/v3/projects/c53868d088284b638b923a07fd036ce6 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:34.652027 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04ee81-c832-4848-be77-9a5df83fb34e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:34.660546 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:34.758548 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04ee81-c832-4848-be77-9a5df83fb34e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:34.792833 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb6131-12c7-4edd-8e55-0f11f3780247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:34.793429 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb6131-12c7-4edd-8e55-0f11f3780247 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:34.793710 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb6131-12c7-4edd-8e55-0f11f3780247 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:34.793977 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb6131-12c7-4edd-8e55-0f11f3780247 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:34.794691 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eb6131-12c7-4edd-8e55-0f11f3780247 None admin] RBAC: Adding query filter params (role.id=bffbe227cda1418cbb58394ddff6f861, scope.project.id=c53868d088284b638b923a07fd036ce6, user.id=bb7e01843be144bbbe0939ad5f7bd79f) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:34.795004 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eb6131-12c7-4edd-8e55-0f11f3780247 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:34.796462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eb6131-12c7-4edd-8e55-0f11f3780247 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:34.802308 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 550/1106] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:24:34 2022] GET /identity/v3/role_assignments?user.id=bb7e01843be144bbbe0939ad5f7bd79f&scope.project.id=c53868d088284b638b923a07fd036ce6&role.id=bffbe227cda1418cbb58394ddff6f861 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:36.833668 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9539585-2580-4aba-a885-03f0717ec96a None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:36.834196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9539585-2580-4aba-a885-03f0717ec96a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:36.834196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9539585-2580-4aba-a885-03f0717ec96a None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:36.835912 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 557/1107] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:36.841223 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62dada-317a-4f80-9b7b-98207ec42dba None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:36.841629 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62dada-317a-4f80-9b7b-98207ec42dba None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:36.841770 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62dada-317a-4f80-9b7b-98207ec42dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:36.860160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7a62dada-317a-4f80-9b7b-98207ec42dba None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:36.861345 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7a62dada-317a-4f80-9b7b-98207ec42dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:36.921982 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:36.926379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 551/1108] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:36.951759 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9539585-2580-4aba-a885-03f0717ec96a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:36.994441 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:37.051662 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9539585-2580-4aba-a885-03f0717ec96a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:37.083831 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1b8c4-c5d5-41ab-9a51-c4a42f23af1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:37.084392 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1b8c4-c5d5-41ab-9a51-c4a42f23af1e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:37.084668 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1b8c4-c5d5-41ab-9a51-c4a42f23af1e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:37.085500 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1b8c4-c5d5-41ab-9a51-c4a42f23af1e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:37.086754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1b8c4-c5d5-41ab-9a51-c4a42f23af1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:37.088184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1b8c4-c5d5-41ab-9a51-c4a42f23af1e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:37.089730 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 558/1109] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:24:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:37.095587 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62dada-317a-4f80-9b7b-98207ec42dba None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:37.192666 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62dada-317a-4f80-9b7b-98207ec42dba None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:37.225810 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de876fd-c5b0-49f7-a7b6-b99b70a5748d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:37.226405 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de876fd-c5b0-49f7-a7b6-b99b70a5748d None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:37.226681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de876fd-c5b0-49f7-a7b6-b99b70a5748d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:37.226949 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de876fd-c5b0-49f7-a7b6-b99b70a5748d None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:37.227607 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de876fd-c5b0-49f7-a7b6-b99b70a5748d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:37.229015 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de876fd-c5b0-49f7-a7b6-b99b70a5748d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:37.335991 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 552/1110] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 12:24:37 2022] POST /identity/v3/users => generated 286 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:24:39.354919 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e398a0-9cb9-4b86-8b6a-00a79267b1b9 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:39.355488 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e398a0-9cb9-4b86-8b6a-00a79267b1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:39.355488 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e398a0-9cb9-4b86-8b6a-00a79267b1b9 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:39.356832 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 559/1111] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:39.361865 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4957b8e-64af-43d2-9f24-a110fd6b50a9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:39.362263 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4957b8e-64af-43d2-9f24-a110fd6b50a9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:39.362427 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4957b8e-64af-43d2-9f24-a110fd6b50a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:39.380414 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d4957b8e-64af-43d2-9f24-a110fd6b50a9 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:39.381605 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d4957b8e-64af-43d2-9f24-a110fd6b50a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:39.431864 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:39.448036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 553/1112] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:39.471193 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e398a0-9cb9-4b86-8b6a-00a79267b1b9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:39.570075 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e398a0-9cb9-4b86-8b6a-00a79267b1b9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:39.579767 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:39.604625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1365a-72c6-43ba-9f28-71ba394ee94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:39.605248 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1365a-72c6-43ba-9f28-71ba394ee94b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:39.605529 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1365a-72c6-43ba-9f28-71ba394ee94b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:39.605805 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1365a-72c6-43ba-9f28-71ba394ee94b None admin] PATH_INFO: `/v3/roles/0f58d2782a2840648bbbc2ab26ec02df` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:39.607212 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d1365a-72c6-43ba-9f28-71ba394ee94b None admin] RBAC: Authorizing `identity:get_role(role_id=0f58d2782a2840648bbbc2ab26ec02df)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:39.608629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d1365a-72c6-43ba-9f28-71ba394ee94b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:39.610376 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 560/1113] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:24:39 2022] GET /identity/v3/roles/0f58d2782a2840648bbbc2ab26ec02df => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:39.616032 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4957b8e-64af-43d2-9f24-a110fd6b50a9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:39.713735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4957b8e-64af-43d2-9f24-a110fd6b50a9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:39.746464 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452abbd4-9b62-459f-8972-aef55eb8cb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:39.747363 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452abbd4-9b62-459f-8972-aef55eb8cb37 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:39.747639 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452abbd4-9b62-459f-8972-aef55eb8cb37 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:39.747908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452abbd4-9b62-459f-8972-aef55eb8cb37 None admin] PATH_INFO: `/v3/users/22ce1468664249ea82ffe5fa0dfea961` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:39.761396 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452abbd4-9b62-459f-8972-aef55eb8cb37 None admin] RBAC: Authorizing `identity:get_user(user_id=22ce1468664249ea82ffe5fa0dfea961)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:39.762810 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452abbd4-9b62-459f-8972-aef55eb8cb37 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:39.764392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 554/1114] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:24:39 2022] GET /identity/v3/users/22ce1468664249ea82ffe5fa0dfea961 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:39.770019 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1365a-72c6-43ba-9f28-71ba394ee94b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:39.868362 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1365a-72c6-43ba-9f28-71ba394ee94b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:39.903083 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b3da4-2b4c-4a6a-8fa2-097526da9c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:39.903704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b3da4-2b4c-4a6a-8fa2-097526da9c0f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:39.903981 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b3da4-2b4c-4a6a-8fa2-097526da9c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:39.904250 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b3da4-2b4c-4a6a-8fa2-097526da9c0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:39.946868 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:39.968757 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560b3da4-2b4c-4a6a-8fa2-097526da9c0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:39.970184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560b3da4-2b4c-4a6a-8fa2-097526da9c0f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:40.064772 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 561/1115] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:24:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:24:40.071371 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452abbd4-9b62-459f-8972-aef55eb8cb37 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:40.116829 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:40.169868 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452abbd4-9b62-459f-8972-aef55eb8cb37 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:40.202220 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5fd282-1ea2-416f-83f0-c9da488bb7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:40.202833 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fd282-1ea2-416f-83f0-c9da488bb7d0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:40.213025 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fd282-1ea2-416f-83f0-c9da488bb7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:40.213025 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fd282-1ea2-416f-83f0-c9da488bb7d0 None admin] PATH_INFO: `/v3/projects/c53868d088284b638b923a07fd036ce6` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:40.213025 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fd282-1ea2-416f-83f0-c9da488bb7d0 None admin] RBAC: Authorizing `identity:get_project(project_id=c53868d088284b638b923a07fd036ce6)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:40.213025 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fd282-1ea2-416f-83f0-c9da488bb7d0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:40.213025 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 555/1116] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:24:40 2022] GET /identity/v3/projects/c53868d088284b638b923a07fd036ce6 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:40.214344 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b3da4-2b4c-4a6a-8fa2-097526da9c0f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:40.299167 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:40.321249 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b3da4-2b4c-4a6a-8fa2-097526da9c0f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:40.354594 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f282e-d17d-406b-9736-a4216314e376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:40.355324 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f282e-d17d-406b-9736-a4216314e376 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:40.355845 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f282e-d17d-406b-9736-a4216314e376 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:40.355845 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f282e-d17d-406b-9736-a4216314e376 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:40.356544 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f282e-d17d-406b-9736-a4216314e376 None admin] RBAC: Adding query filter params (role.id=0f58d2782a2840648bbbc2ab26ec02df, scope.project.id=c53868d088284b638b923a07fd036ce6, user.id=22ce1468664249ea82ffe5fa0dfea961) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:40.356846 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f282e-d17d-406b-9736-a4216314e376 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:40.358240 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f282e-d17d-406b-9736-a4216314e376 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:40.363405 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 562/1117] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:24:40 2022] GET /identity/v3/role_assignments?user.id=22ce1468664249ea82ffe5fa0dfea961&scope.project.id=c53868d088284b638b923a07fd036ce6&role.id=0f58d2782a2840648bbbc2ab26ec02df => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:42.336579 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89594c07-576a-4237-a014-9bb01f5188ea None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:42.337059 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89594c07-576a-4237-a014-9bb01f5188ea None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:42.337155 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89594c07-576a-4237-a014-9bb01f5188ea None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:42.338468 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 556/1118] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:42.343515 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c352f8-08ed-42b9-b039-f9d3ee0a5759 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:42.343904 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c352f8-08ed-42b9-b039-f9d3ee0a5759 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:42.344064 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c352f8-08ed-42b9-b039-f9d3ee0a5759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:42.362462 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e3c352f8-08ed-42b9-b039-f9d3ee0a5759 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:42.363706 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e3c352f8-08ed-42b9-b039-f9d3ee0a5759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:42.429901 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 563/1119] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:42.451552 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89594c07-576a-4237-a014-9bb01f5188ea None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:42.551356 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89594c07-576a-4237-a014-9bb01f5188ea None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:42.563835 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:42.583633 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16daa95-29e8-458b-a395-9be86dce4cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:42.586140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16daa95-29e8-458b-a395-9be86dce4cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:42.586140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16daa95-29e8-458b-a395-9be86dce4cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:42.586140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16daa95-29e8-458b-a395-9be86dce4cb0 None admin] PATH_INFO: `/v3/roles/0f58d2782a2840648bbbc2ab26ec02df` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:42.586828 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16daa95-29e8-458b-a395-9be86dce4cb0 None admin] RBAC: Authorizing `identity:get_role(role_id=0f58d2782a2840648bbbc2ab26ec02df)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:42.588219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16daa95-29e8-458b-a395-9be86dce4cb0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:42.589507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 557/1120] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:24:42 2022] GET /identity/v3/roles/0f58d2782a2840648bbbc2ab26ec02df => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:42.595215 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c352f8-08ed-42b9-b039-f9d3ee0a5759 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:42.663317 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:42.693291 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c352f8-08ed-42b9-b039-f9d3ee0a5759 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:42.726915 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff10c1d2-abc3-450a-9809-f876fc52712f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:42.727568 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10c1d2-abc3-450a-9809-f876fc52712f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:42.727850 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10c1d2-abc3-450a-9809-f876fc52712f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:42.728123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10c1d2-abc3-450a-9809-f876fc52712f None admin] PATH_INFO: `/v3/users/22ce1468664249ea82ffe5fa0dfea961` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:42.729246 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff10c1d2-abc3-450a-9809-f876fc52712f None admin] RBAC: Authorizing `identity:get_user(user_id=22ce1468664249ea82ffe5fa0dfea961)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:42.730673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff10c1d2-abc3-450a-9809-f876fc52712f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:42.732275 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 564/1121] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:24:42 2022] GET /identity/v3/users/22ce1468664249ea82ffe5fa0dfea961 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:42.737985 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16daa95-29e8-458b-a395-9be86dce4cb0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:42.835317 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16daa95-29e8-458b-a395-9be86dce4cb0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:42.867885 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838e73d-bd9e-4570-b918-63fbaeb729c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:42.868499 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838e73d-bd9e-4570-b918-63fbaeb729c4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:42.868777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838e73d-bd9e-4570-b918-63fbaeb729c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:42.869046 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838e73d-bd9e-4570-b918-63fbaeb729c4 None admin] PATH_INFO: `/v3/projects/c53868d088284b638b923a07fd036ce6` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:42.870153 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7838e73d-bd9e-4570-b918-63fbaeb729c4 None admin] RBAC: Authorizing `identity:get_project(project_id=c53868d088284b638b923a07fd036ce6)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:42.871607 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7838e73d-bd9e-4570-b918-63fbaeb729c4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:42.873468 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 558/1122] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:24:42 2022] GET /identity/v3/projects/c53868d088284b638b923a07fd036ce6 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:42.879340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff10c1d2-abc3-450a-9809-f876fc52712f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:42.977152 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff10c1d2-abc3-450a-9809-f876fc52712f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:43.010394 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23ee1bc-6495-4b72-9ab7-dbaa2676dbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:43.011032 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ee1bc-6495-4b72-9ab7-dbaa2676dbc6 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:43.011336 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ee1bc-6495-4b72-9ab7-dbaa2676dbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:43.011606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ee1bc-6495-4b72-9ab7-dbaa2676dbc6 None admin] PATH_INFO: `/v3/projects/c53868d088284b638b923a07fd036ce6/users/22ce1468664249ea82ffe5fa0dfea961/roles/0f58d2782a2840648bbbc2ab26ec02df` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:43.013711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23ee1bc-6495-4b72-9ab7-dbaa2676dbc6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c53868d088284b638b923a07fd036ce6, user_id=22ce1468664249ea82ffe5fa0dfea961, role_id=0f58d2782a2840648bbbc2ab26ec02df)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:43.015114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23ee1bc-6495-4b72-9ab7-dbaa2676dbc6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:43.090170 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:43.095079 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 565/1123] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 12:24:42 2022] PUT /identity/v3/projects/c53868d088284b638b923a07fd036ce6/users/22ce1468664249ea82ffe5fa0dfea961/roles/0f58d2782a2840648bbbc2ab26ec02df => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:24:45.058616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24038abc-d6cf-47d0-b2a1-ebf59918f06c None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:45.059199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24038abc-d6cf-47d0-b2a1-ebf59918f06c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:45.059199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24038abc-d6cf-47d0-b2a1-ebf59918f06c None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:45.060542 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 559/1124] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:45.065586 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6bd1e-0695-4bee-a17e-9380a6664c76 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:45.066025 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6bd1e-0695-4bee-a17e-9380a6664c76 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:45.066127 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6bd1e-0695-4bee-a17e-9380a6664c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:45.085718 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-abe6bd1e-0695-4bee-a17e-9380a6664c76 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:45.086920 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-abe6bd1e-0695-4bee-a17e-9380a6664c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:45.151630 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 566/1125] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:45.174994 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24038abc-d6cf-47d0-b2a1-ebf59918f06c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:45.223503 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:45.275711 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24038abc-d6cf-47d0-b2a1-ebf59918f06c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:45.309184 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78342b39-edf0-4c8d-95c8-adf4cf3e731c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:45.309750 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78342b39-edf0-4c8d-95c8-adf4cf3e731c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:45.310032 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78342b39-edf0-4c8d-95c8-adf4cf3e731c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:45.310309 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78342b39-edf0-4c8d-95c8-adf4cf3e731c None admin] PATH_INFO: `/v3/roles/0f58d2782a2840648bbbc2ab26ec02df` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:45.311721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78342b39-edf0-4c8d-95c8-adf4cf3e731c None admin] RBAC: Authorizing `identity:get_role(role_id=0f58d2782a2840648bbbc2ab26ec02df)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:45.313083 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78342b39-edf0-4c8d-95c8-adf4cf3e731c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:45.314389 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 560/1126] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:24:45 2022] GET /identity/v3/roles/0f58d2782a2840648bbbc2ab26ec02df => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:45.320380 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6bd1e-0695-4bee-a17e-9380a6664c76 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:45.416061 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6bd1e-0695-4bee-a17e-9380a6664c76 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:45.440865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:45.448578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58a07b-5284-454e-a5e0-2d96d05b2ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:45.461056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58a07b-5284-454e-a5e0-2d96d05b2ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:45.461056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58a07b-5284-454e-a5e0-2d96d05b2ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:45.461056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58a07b-5284-454e-a5e0-2d96d05b2ab5 None admin] PATH_INFO: `/v3/users/22ce1468664249ea82ffe5fa0dfea961` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:45.461056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e58a07b-5284-454e-a5e0-2d96d05b2ab5 None admin] RBAC: Authorizing `identity:get_user(user_id=22ce1468664249ea82ffe5fa0dfea961)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:45.461056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e58a07b-5284-454e-a5e0-2d96d05b2ab5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:45.461056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 567/1127] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:24:45 2022] GET /identity/v3/users/22ce1468664249ea82ffe5fa0dfea961 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:45.461573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78342b39-edf0-4c8d-95c8-adf4cf3e731c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:45.556507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78342b39-edf0-4c8d-95c8-adf4cf3e731c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:45.571791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:45.589144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379bf5b-013b-48ef-a84e-7a8d23f6832c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:45.589775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379bf5b-013b-48ef-a84e-7a8d23f6832c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:45.590054 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379bf5b-013b-48ef-a84e-7a8d23f6832c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:45.590325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379bf5b-013b-48ef-a84e-7a8d23f6832c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:45.654261 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379bf5b-013b-48ef-a84e-7a8d23f6832c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:45.655721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379bf5b-013b-48ef-a84e-7a8d23f6832c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:45.749598 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 561/1128] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:24:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:24:45.755481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58a07b-5284-454e-a5e0-2d96d05b2ab5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:45.852241 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58a07b-5284-454e-a5e0-2d96d05b2ab5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:45.886364 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3144cc20-42cc-4a87-bf4a-1da0ac3d2603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:45.886987 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144cc20-42cc-4a87-bf4a-1da0ac3d2603 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:45.887297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144cc20-42cc-4a87-bf4a-1da0ac3d2603 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:45.887573 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144cc20-42cc-4a87-bf4a-1da0ac3d2603 None admin] PATH_INFO: `/v3/projects/c53868d088284b638b923a07fd036ce6` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:45.888685 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3144cc20-42cc-4a87-bf4a-1da0ac3d2603 None admin] RBAC: Authorizing `identity:get_project(project_id=c53868d088284b638b923a07fd036ce6)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:45.890100 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3144cc20-42cc-4a87-bf4a-1da0ac3d2603 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:45.891755 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 568/1129] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:24:45 2022] GET /identity/v3/projects/c53868d088284b638b923a07fd036ce6 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:45.898561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379bf5b-013b-48ef-a84e-7a8d23f6832c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:45.916403 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:45.994946 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379bf5b-013b-48ef-a84e-7a8d23f6832c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:46.027197 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58690fad-ff06-43b6-837d-8c5cbd1b3359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:46.027799 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58690fad-ff06-43b6-837d-8c5cbd1b3359 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:46.028074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58690fad-ff06-43b6-837d-8c5cbd1b3359 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:46.028345 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58690fad-ff06-43b6-837d-8c5cbd1b3359 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:46.029049 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58690fad-ff06-43b6-837d-8c5cbd1b3359 None admin] RBAC: Adding query filter params (role.id=0f58d2782a2840648bbbc2ab26ec02df, scope.project.id=c53868d088284b638b923a07fd036ce6, user.id=22ce1468664249ea82ffe5fa0dfea961) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:46.029356 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58690fad-ff06-43b6-837d-8c5cbd1b3359 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:46.030757 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58690fad-ff06-43b6-837d-8c5cbd1b3359 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:46.036258 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 562/1130] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:24:45 2022] GET /identity/v3/role_assignments?user.id=22ce1468664249ea82ffe5fa0dfea961&scope.project.id=c53868d088284b638b923a07fd036ce6&role.id=0f58d2782a2840648bbbc2ab26ec02df => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:48.046240 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bebbe8b-c157-452d-9313-2fc18c83e8c9 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:48.046797 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bebbe8b-c157-452d-9313-2fc18c83e8c9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:48.046797 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bebbe8b-c157-452d-9313-2fc18c83e8c9 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:48.048225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 569/1131] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:48.053330 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f8793-b027-4f16-b629-d7ae492640bf None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:48.053740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f8793-b027-4f16-b629-d7ae492640bf None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:48.053986 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f8793-b027-4f16-b629-d7ae492640bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:48.072088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-572f8793-b027-4f16-b629-d7ae492640bf None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:48.073281 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-572f8793-b027-4f16-b629-d7ae492640bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:48.162724 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 563/1132] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:48.164065 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bebbe8b-c157-452d-9313-2fc18c83e8c9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:48.230805 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:48.268828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bebbe8b-c157-452d-9313-2fc18c83e8c9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:48.306187 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed26b64-2f86-4efb-9422-db03e9561afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:48.306759 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed26b64-2f86-4efb-9422-db03e9561afb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:48.307036 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed26b64-2f86-4efb-9422-db03e9561afb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:48.307334 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed26b64-2f86-4efb-9422-db03e9561afb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:48.308430 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed26b64-2f86-4efb-9422-db03e9561afb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:48.309832 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed26b64-2f86-4efb-9422-db03e9561afb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:48.311411 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 570/1133] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:24:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:48.317284 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f8793-b027-4f16-b629-d7ae492640bf None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:48.329431 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:48.414862 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f8793-b027-4f16-b629-d7ae492640bf None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:48.448243 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21baa6bd-6a53-436a-93f3-87174bccded2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:48.448839 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21baa6bd-6a53-436a-93f3-87174bccded2 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:48.449117 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21baa6bd-6a53-436a-93f3-87174bccded2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:48.449386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21baa6bd-6a53-436a-93f3-87174bccded2 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:48.450026 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21baa6bd-6a53-436a-93f3-87174bccded2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:48.451482 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21baa6bd-6a53-436a-93f3-87174bccded2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:48.564612 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 564/1134] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 10 12:24:48 2022] POST /identity/v3/projects => generated 311 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:24:50.561309 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72173136-02e4-4340-af96-dceb9ddf9d4a None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:50.561825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72173136-02e4-4340-af96-dceb9ddf9d4a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:50.561825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72173136-02e4-4340-af96-dceb9ddf9d4a None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:50.563116 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 571/1135] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:50.568179 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5c5cb-f279-4594-9f8f-23d762511874 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:50.568560 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5c5cb-f279-4594-9f8f-23d762511874 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:50.568726 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5c5cb-f279-4594-9f8f-23d762511874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:50.587576 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-55d5c5cb-f279-4594-9f8f-23d762511874 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:50.588752 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-55d5c5cb-f279-4594-9f8f-23d762511874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:50.643851 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:50.654457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 565/1136] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:50.678672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72173136-02e4-4340-af96-dceb9ddf9d4a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:50.777057 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72173136-02e4-4340-af96-dceb9ddf9d4a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:50.805568 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:50.810191 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f71fd-d22a-43ed-9038-fcc0babf1ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:50.811260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f71fd-d22a-43ed-9038-fcc0babf1ece None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:50.811260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f71fd-d22a-43ed-9038-fcc0babf1ece None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:50.811527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f71fd-d22a-43ed-9038-fcc0babf1ece None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:50.812566 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f71fd-d22a-43ed-9038-fcc0babf1ece None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:50.814052 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f71fd-d22a-43ed-9038-fcc0babf1ece None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:50.815613 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 572/1137] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:24:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:50.821370 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d5c5cb-f279-4594-9f8f-23d762511874 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:50.917338 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d5c5cb-f279-4594-9f8f-23d762511874 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:50.950353 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97bd138-5109-4155-b9c3-a5758bf1f446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:50.950946 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97bd138-5109-4155-b9c3-a5758bf1f446 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:50.951252 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97bd138-5109-4155-b9c3-a5758bf1f446 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:50.951531 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97bd138-5109-4155-b9c3-a5758bf1f446 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:50.952173 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97bd138-5109-4155-b9c3-a5758bf1f446 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:50.953589 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97bd138-5109-4155-b9c3-a5758bf1f446 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:51.053134 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 566/1138] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 12:24:50 2022] POST /identity/v3/users => generated 286 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:24:53.096238 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81788dc-8f70-4ed6-a11b-41fb76545184 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:53.096787 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81788dc-8f70-4ed6-a11b-41fb76545184 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:53.096787 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81788dc-8f70-4ed6-a11b-41fb76545184 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:53.098101 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 573/1139] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:53.103819 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fc35f-fd20-4462-b946-5dca74f961ae None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:53.104214 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fc35f-fd20-4462-b946-5dca74f961ae None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:53.104372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fc35f-fd20-4462-b946-5dca74f961ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:53.122882 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-238fc35f-fd20-4462-b946-5dca74f961ae None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:53.124116 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-238fc35f-fd20-4462-b946-5dca74f961ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:53.167352 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:53.189119 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 567/1140] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:53.213015 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81788dc-8f70-4ed6-a11b-41fb76545184 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:53.310556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81788dc-8f70-4ed6-a11b-41fb76545184 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:53.343307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428d53c-cc9b-425e-aef9-6b7460dfbcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:53.343923 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428d53c-cc9b-425e-aef9-6b7460dfbcbc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:53.344198 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428d53c-cc9b-425e-aef9-6b7460dfbcbc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:53.344467 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428d53c-cc9b-425e-aef9-6b7460dfbcbc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:53.354227 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428d53c-cc9b-425e-aef9-6b7460dfbcbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:24:53.358727 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-8428d53c-cc9b-425e-aef9-6b7460dfbcbc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:24:53.358727 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:53.358727 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:53.358727 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:53.358727 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:53.358727 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:53.358727 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:53.358727 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:53.358727 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:53.358727 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:53.358727 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:53.358727 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:53.358727 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:24:53.358727 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:24:53.358727 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:24:53.358727 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:24:53.359903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:24:53.359903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:24:53.359903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:24:53.359903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:24:53.359903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:53.359903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:53.359903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:53.359903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:53.359903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:53.359903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:53.359903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:53.359903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:53.359903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:53.359903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:53.359903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:53.359903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:53.361162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:53.361162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:53.361162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:53.361162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:53.361162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:24:53.361162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:24:53.361162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:24:53.361162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:24:53.361162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:24:53.361162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:24:53.361162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:24:53.361162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:53.361162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 574/1141] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:24:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:53.365932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238fc35f-fd20-4462-b946-5dca74f961ae None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:53.464043 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238fc35f-fd20-4462-b946-5dca74f961ae None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:53.496344 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8340831-8969-4149-9123-6282af9bdb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:53.496945 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8340831-8969-4149-9123-6282af9bdb8d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:53.497486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8340831-8969-4149-9123-6282af9bdb8d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:53.497761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8340831-8969-4149-9123-6282af9bdb8d None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:53.498416 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8340831-8969-4149-9123-6282af9bdb8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:53.498721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8340831-8969-4149-9123-6282af9bdb8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:53.500109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8340831-8969-4149-9123-6282af9bdb8d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:53.508394 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 568/1142] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:24:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:53.514089 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428d53c-cc9b-425e-aef9-6b7460dfbcbc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:53.573154 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:53.611706 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428d53c-cc9b-425e-aef9-6b7460dfbcbc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:53.643862 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38edcd-dd47-4e6e-b3f4-e013d4d00cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:53.644449 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38edcd-dd47-4e6e-b3f4-e013d4d00cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:53.644726 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38edcd-dd47-4e6e-b3f4-e013d4d00cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:53.644996 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38edcd-dd47-4e6e-b3f4-e013d4d00cf1 None admin] PATH_INFO: `/v3/users/5b26a12b64b147309ac5703362e42435` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:53.658013 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38edcd-dd47-4e6e-b3f4-e013d4d00cf1 None admin] RBAC: Authorizing `identity:get_user(user_id=5b26a12b64b147309ac5703362e42435)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:53.659460 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38edcd-dd47-4e6e-b3f4-e013d4d00cf1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:53.661028 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 575/1143] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:24:53 2022] GET /identity/v3/users/5b26a12b64b147309ac5703362e42435 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:24:53.666893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8340831-8969-4149-9123-6282af9bdb8d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:53.709527 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:53.765863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8340831-8969-4149-9123-6282af9bdb8d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:53.797699 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad6b66-cb70-43f8-b0af-01920e09f498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:53.798291 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad6b66-cb70-43f8-b0af-01920e09f498 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:53.798573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad6b66-cb70-43f8-b0af-01920e09f498 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:53.798846 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad6b66-cb70-43f8-b0af-01920e09f498 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:53.862932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad6b66-cb70-43f8-b0af-01920e09f498 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:53.864345 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad6b66-cb70-43f8-b0af-01920e09f498 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:53.904458 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:53.962075 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 569/1144] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:24:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:24:53.968087 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38edcd-dd47-4e6e-b3f4-e013d4d00cf1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:54.099095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38edcd-dd47-4e6e-b3f4-e013d4d00cf1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:54.116316 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:54.133615 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42089f92-4423-43d5-9e00-ca61206f4bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:54.134231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42089f92-4423-43d5-9e00-ca61206f4bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:54.134510 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42089f92-4423-43d5-9e00-ca61206f4bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:54.136239 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42089f92-4423-43d5-9e00-ca61206f4bb8 None admin] PATH_INFO: `/v3/projects/b247040963e446469242c20362e460a1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:54.136239 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42089f92-4423-43d5-9e00-ca61206f4bb8 None admin] RBAC: Authorizing `identity:get_project(project_id=b247040963e446469242c20362e460a1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:54.137321 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42089f92-4423-43d5-9e00-ca61206f4bb8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:54.138939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 576/1145] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:24:53 2022] GET /identity/v3/projects/b247040963e446469242c20362e460a1 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:54.145140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad6b66-cb70-43f8-b0af-01920e09f498 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:54.242952 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad6b66-cb70-43f8-b0af-01920e09f498 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:54.276041 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405dca5-179c-4706-96f4-181a06a9d4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:54.276644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405dca5-179c-4706-96f4-181a06a9d4a5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:54.276925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405dca5-179c-4706-96f4-181a06a9d4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:54.277207 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405dca5-179c-4706-96f4-181a06a9d4a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:54.277923 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d405dca5-179c-4706-96f4-181a06a9d4a5 None admin] RBAC: Adding query filter params (role.id=bffbe227cda1418cbb58394ddff6f861, scope.project.id=b247040963e446469242c20362e460a1, user.id=5b26a12b64b147309ac5703362e42435) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:54.278232 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d405dca5-179c-4706-96f4-181a06a9d4a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:54.279704 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d405dca5-179c-4706-96f4-181a06a9d4a5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:54.285055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 570/1146] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:24:54 2022] GET /identity/v3/role_assignments?user.id=5b26a12b64b147309ac5703362e42435&scope.project.id=b247040963e446469242c20362e460a1&role.id=bffbe227cda1418cbb58394ddff6f861 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:56.229134 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f04b0-9f33-4434-b544-fa132d00a52c None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:56.229698 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f04b0-9f33-4434-b544-fa132d00a52c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:56.229698 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f04b0-9f33-4434-b544-fa132d00a52c None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:56.231016 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 577/1147] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:56.236220 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164debc1-81e4-474c-ae75-843839ba6aa0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:56.236635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164debc1-81e4-474c-ae75-843839ba6aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:56.236776 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164debc1-81e4-474c-ae75-843839ba6aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:56.244005 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:56.254681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-164debc1-81e4-474c-ae75-843839ba6aa0 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:56.255929 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-164debc1-81e4-474c-ae75-843839ba6aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:56.322903 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 571/1148] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:56.345109 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f04b0-9f33-4434-b544-fa132d00a52c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:56.444418 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f04b0-9f33-4434-b544-fa132d00a52c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:56.477875 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49cd5f-3086-4521-be0d-608beda2eda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:56.478489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49cd5f-3086-4521-be0d-608beda2eda1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:56.478767 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49cd5f-3086-4521-be0d-608beda2eda1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:56.479038 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49cd5f-3086-4521-be0d-608beda2eda1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:56.488916 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49cd5f-3086-4521-be0d-608beda2eda1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:24:56.493653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-7a49cd5f-3086-4521-be0d-608beda2eda1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:24:56.493653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:56.493653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:56.493653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:56.493653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:56.493653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:56.493653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:56.493653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:56.493653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:56.493653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:56.493653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:56.493653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:56.493653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:24:56.493653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:24:56.493653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:24:56.493653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:24:56.494731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:24:56.494731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:24:56.494731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:24:56.494731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:24:56.494731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:56.494731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:56.494731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:56.494731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:56.494731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:56.494731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:56.494731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:56.494731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:56.494731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:56.494731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:56.494731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:56.494731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:56.495849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:56.495849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:56.495849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:56.495849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:56.495849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:24:56.495849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:24:56.495849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:24:56.495849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:24:56.495849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:24:56.495849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:24:56.495849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:24:56.495849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:24:56.495849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 578/1149] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:24:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:56.500979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164debc1-81e4-474c-ae75-843839ba6aa0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:56.595688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164debc1-81e4-474c-ae75-843839ba6aa0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:56.628072 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f262e2-f474-41b8-ad80-326218bce53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:56.628661 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f262e2-f474-41b8-ad80-326218bce53f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:56.628935 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f262e2-f474-41b8-ad80-326218bce53f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:56.629202 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f262e2-f474-41b8-ad80-326218bce53f None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:56.629849 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f262e2-f474-41b8-ad80-326218bce53f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:56.630150 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f262e2-f474-41b8-ad80-326218bce53f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:56.631513 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f262e2-f474-41b8-ad80-326218bce53f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:56.632894 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:56.639756 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 572/1150] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:24:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:56.645507 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49cd5f-3086-4521-be0d-608beda2eda1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:56.692004 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:56.743651 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49cd5f-3086-4521-be0d-608beda2eda1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:56.776371 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c686075-a10e-478e-8496-5ad644d5033c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:56.776990 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c686075-a10e-478e-8496-5ad644d5033c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:56.777272 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c686075-a10e-478e-8496-5ad644d5033c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:56.777549 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c686075-a10e-478e-8496-5ad644d5033c None admin] PATH_INFO: `/v3/users/5b26a12b64b147309ac5703362e42435` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:56.778674 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c686075-a10e-478e-8496-5ad644d5033c None admin] RBAC: Authorizing `identity:get_user(user_id=5b26a12b64b147309ac5703362e42435)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:56.780122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c686075-a10e-478e-8496-5ad644d5033c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:56.781686 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 579/1151] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:24:56 2022] GET /identity/v3/users/5b26a12b64b147309ac5703362e42435 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:56.787289 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f262e2-f474-41b8-ad80-326218bce53f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:56.883286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f262e2-f474-41b8-ad80-326218bce53f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:56.915162 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b71a2-965b-449b-972a-176d432bdc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:56.915797 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b71a2-965b-449b-972a-176d432bdc3c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:56.916074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b71a2-965b-449b-972a-176d432bdc3c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:56.916347 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b71a2-965b-449b-972a-176d432bdc3c None admin] PATH_INFO: `/v3/projects/b247040963e446469242c20362e460a1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:56.917441 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b71a2-965b-449b-972a-176d432bdc3c None admin] RBAC: Authorizing `identity:get_project(project_id=b247040963e446469242c20362e460a1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:56.918851 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b71a2-965b-449b-972a-176d432bdc3c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:56.920460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 573/1152] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:24:56 2022] GET /identity/v3/projects/b247040963e446469242c20362e460a1 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:56.926370 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c686075-a10e-478e-8496-5ad644d5033c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:57.024511 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c686075-a10e-478e-8496-5ad644d5033c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:57.037959 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:57.057911 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b68e2a-e8ac-4163-be23-babcacabd9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:57.058558 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b68e2a-e8ac-4163-be23-babcacabd9e6 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:57.058883 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b68e2a-e8ac-4163-be23-babcacabd9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:57.059168 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b68e2a-e8ac-4163-be23-babcacabd9e6 None admin] PATH_INFO: `/v3/projects/b247040963e446469242c20362e460a1/users/5b26a12b64b147309ac5703362e42435/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:57.061088 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b68e2a-e8ac-4163-be23-babcacabd9e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=b247040963e446469242c20362e460a1, user_id=5b26a12b64b147309ac5703362e42435, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:57.062502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b68e2a-e8ac-4163-be23-babcacabd9e6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:57.158195 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 580/1153] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 12:24:56 2022] PUT /identity/v3/projects/b247040963e446469242c20362e460a1/users/5b26a12b64b147309ac5703362e42435/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:24:59.135002 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b61d33-c45c-415b-bb97-2d6ecd738fd5 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:59.135666 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b61d33-c45c-415b-bb97-2d6ecd738fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:59.135666 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b61d33-c45c-415b-bb97-2d6ecd738fd5 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:59.136920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 574/1154] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:59.142037 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49409ef2-846e-47e3-9264-943dde4d339b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:59.142449 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49409ef2-846e-47e3-9264-943dde4d339b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:59.142595 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49409ef2-846e-47e3-9264-943dde4d339b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:59.161423 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-49409ef2-846e-47e3-9264-943dde4d339b None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:59.162620 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-49409ef2-846e-47e3-9264-943dde4d339b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:59.228472 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 581/1155] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:24:59.251496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b61d33-c45c-415b-bb97-2d6ecd738fd5 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:59.312983 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:59.350596 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b61d33-c45c-415b-bb97-2d6ecd738fd5 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:59.383190 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9107f47-0228-47f9-9404-e8cd247ff249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:59.383816 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9107f47-0228-47f9-9404-e8cd247ff249 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:59.384094 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9107f47-0228-47f9-9404-e8cd247ff249 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:59.384367 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9107f47-0228-47f9-9404-e8cd247ff249 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:59.394504 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9107f47-0228-47f9-9404-e8cd247ff249 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:24:59.399791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d9107f47-0228-47f9-9404-e8cd247ff249 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:24:59.399791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:24:59.399791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:24:59.399791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:24:59.399791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:24:59.399791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:24:59.399791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:24:59.399791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:24:59.399791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:24:59.399791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:24:59.399791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:24:59.399791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:24:59.399791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:24:59.399791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:24:59.399791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:24:59.399791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:24:59.400905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:24:59.400905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:24:59.400905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:24:59.400905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:24:59.400905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:24:59.400905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:24:59.400905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:24:59.400905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:24:59.400905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:24:59.400905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:24:59.400905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:24:59.400905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:24:59.400905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:24:59.400905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:24:59.400905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:24:59.400905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:24:59.401942 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:24:59.401942 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:24:59.401942 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:24:59.401942 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:24:59.401942 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:24:59.401942 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:24:59.401942 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:24:59.401942 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:24:59.401942 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:24:59.401942 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:24:59.401942 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:24:59.401942 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:24:59.401942 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 575/1156] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:24:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:24:59.407040 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49409ef2-846e-47e3-9264-943dde4d339b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:59.503366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49409ef2-846e-47e3-9264-943dde4d339b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:59.505558 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:59.535996 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de9a07-4687-48b5-ace0-70b3446c57fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:59.536585 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de9a07-4687-48b5-ace0-70b3446c57fe None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:59.536857 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de9a07-4687-48b5-ace0-70b3446c57fe None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:59.537123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de9a07-4687-48b5-ace0-70b3446c57fe None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:59.537770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de9a07-4687-48b5-ace0-70b3446c57fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:24:59.538069 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de9a07-4687-48b5-ace0-70b3446c57fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:59.540415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de9a07-4687-48b5-ace0-70b3446c57fe None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:59.548168 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 582/1157] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:24:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:59.553831 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9107f47-0228-47f9-9404-e8cd247ff249 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:59.652906 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9107f47-0228-47f9-9404-e8cd247ff249 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:59.675162 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:59.685388 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1a792-50f6-41ad-93a3-8668d4b3dfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:59.686008 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1a792-50f6-41ad-93a3-8668d4b3dfca None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:59.687809 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1a792-50f6-41ad-93a3-8668d4b3dfca None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:59.687809 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1a792-50f6-41ad-93a3-8668d4b3dfca None admin] PATH_INFO: `/v3/users/5b26a12b64b147309ac5703362e42435` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:59.687809 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1a792-50f6-41ad-93a3-8668d4b3dfca None admin] RBAC: Authorizing `identity:get_user(user_id=5b26a12b64b147309ac5703362e42435)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:59.689060 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1a792-50f6-41ad-93a3-8668d4b3dfca None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:59.690605 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 576/1158] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:24:59 2022] GET /identity/v3/users/5b26a12b64b147309ac5703362e42435 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:24:59.696355 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de9a07-4687-48b5-ace0-70b3446c57fe None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:59.793882 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de9a07-4687-48b5-ace0-70b3446c57fe None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:59.825464 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80bc06-491f-4705-aeb2-d8831b9250a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:59.826047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80bc06-491f-4705-aeb2-d8831b9250a1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:59.826324 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80bc06-491f-4705-aeb2-d8831b9250a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:59.826595 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80bc06-491f-4705-aeb2-d8831b9250a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:59.858421 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:59.892486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80bc06-491f-4705-aeb2-d8831b9250a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:59.894443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80bc06-491f-4705-aeb2-d8831b9250a1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:59.989154 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 583/1159] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:24:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:24:59.995857 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1a792-50f6-41ad-93a3-8668d4b3dfca None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:00.094618 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1a792-50f6-41ad-93a3-8668d4b3dfca None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:00.128012 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579ff4c8-2aa5-41fc-ac37-da9dd84fb6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:00.128632 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ff4c8-2aa5-41fc-ac37-da9dd84fb6d9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:00.128912 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ff4c8-2aa5-41fc-ac37-da9dd84fb6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:00.129187 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ff4c8-2aa5-41fc-ac37-da9dd84fb6d9 None admin] PATH_INFO: `/v3/projects/b247040963e446469242c20362e460a1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:00.130285 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579ff4c8-2aa5-41fc-ac37-da9dd84fb6d9 None admin] RBAC: Authorizing `identity:get_project(project_id=b247040963e446469242c20362e460a1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:00.131740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579ff4c8-2aa5-41fc-ac37-da9dd84fb6d9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:00.133911 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 577/1160] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:24:59 2022] GET /identity/v3/projects/b247040963e446469242c20362e460a1 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:25:00.140049 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80bc06-491f-4705-aeb2-d8831b9250a1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:00.202584 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:00.236287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80bc06-491f-4705-aeb2-d8831b9250a1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:00.269561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09621c45-61b2-4ec8-8e8e-ed5133b50180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:00.270167 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09621c45-61b2-4ec8-8e8e-ed5133b50180 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:00.270448 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09621c45-61b2-4ec8-8e8e-ed5133b50180 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:00.270722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09621c45-61b2-4ec8-8e8e-ed5133b50180 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:00.271475 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09621c45-61b2-4ec8-8e8e-ed5133b50180 None admin] RBAC: Adding query filter params (role.id=bffbe227cda1418cbb58394ddff6f861, scope.project.id=b247040963e446469242c20362e460a1, user.id=5b26a12b64b147309ac5703362e42435) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:25:00.271788 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09621c45-61b2-4ec8-8e8e-ed5133b50180 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:00.274231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09621c45-61b2-4ec8-8e8e-ed5133b50180 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:00.281369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 584/1161] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:25:00 2022] GET /identity/v3/role_assignments?user.id=5b26a12b64b147309ac5703362e42435&scope.project.id=b247040963e446469242c20362e460a1&role.id=bffbe227cda1418cbb58394ddff6f861 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:25:02.294856 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45fd47-571a-4adf-b447-21b95f563cec None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:02.295290 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45fd47-571a-4adf-b447-21b95f563cec None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:02.295455 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45fd47-571a-4adf-b447-21b95f563cec None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:02.296801 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 578/1162] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:25:02.301889 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbe515-6c0d-4768-b9de-1f2a95299bbf None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:02.302325 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbe515-6c0d-4768-b9de-1f2a95299bbf None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:02.302448 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbe515-6c0d-4768-b9de-1f2a95299bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:02.321204 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-00fbe515-6c0d-4768-b9de-1f2a95299bbf None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:25:02.322395 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-00fbe515-6c0d-4768-b9de-1f2a95299bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:02.388027 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 585/1163] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:25:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:25:02.410151 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45fd47-571a-4adf-b447-21b95f563cec None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:02.469175 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:02.509550 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45fd47-571a-4adf-b447-21b95f563cec None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:02.542796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b8c7b-b9c2-4d75-a2c3-044d6894b9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:02.543366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b8c7b-b9c2-4d75-a2c3-044d6894b9b9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:02.543646 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b8c7b-b9c2-4d75-a2c3-044d6894b9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:02.543920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b8c7b-b9c2-4d75-a2c3-044d6894b9b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:02.608493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b8c7b-b9c2-4d75-a2c3-044d6894b9b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:02.609923 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b8c7b-b9c2-4d75-a2c3-044d6894b9b9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:02.665187 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:02.705433 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 579/1164] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:25:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:25:02.711492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fbe515-6c0d-4768-b9de-1f2a95299bbf None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:02.767560 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:02.807077 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fbe515-6c0d-4768-b9de-1f2a95299bbf None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:02.839829 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae0008f-678b-4600-a34b-24766b876cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:02.840449 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0008f-678b-4600-a34b-24766b876cde None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:02.840728 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0008f-678b-4600-a34b-24766b876cde None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:02.841000 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0008f-678b-4600-a34b-24766b876cde None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:02.847386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae0008f-678b-4600-a34b-24766b876cde None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:02.848820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae0008f-678b-4600-a34b-24766b876cde None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:02.860222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-0ae0008f-678b-4600-a34b-24766b876cde None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 10 12:25:02.860222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:25:02.860222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 10 12:25:02.860222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 10 12:25:02.860222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:25:02.860222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:25:02.860222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:25:02.860222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:25:02.860222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 10 12:25:02.860222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:25:02.860222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:25:02.860222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:25:02.860222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:25:02.860222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:25:02.860222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:25:02.860222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:25:02.860222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:25:02.861458 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:25:02.861458 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:25:02.861458 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:25:02.861458 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:25:02.861458 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:25:02.861458 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:25:02.861458 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:25:02.861458 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:25:02.861458 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:25:02.861458 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:25:02.861458 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:25:02.861458 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:25:02.861458 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:25:02.861458 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:25:02.861458 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:25:02.861458 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:25:02.862609 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:25:02.862609 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:25:02.862609 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:25:02.862609 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:25:02.862609 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:25:02.862609 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:25:02.862609 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:25:02.862609 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:25:02.862609 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:25:02.862609 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:25:02.862609 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 10 12:25:02.862609 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:25:02.862609 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 10 12:25:02.862609 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:25:02.862609 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 586/1165] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 10 12:25:02 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:25:02.868111 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b8c7b-b9c2-4d75-a2c3-044d6894b9b9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:02.964839 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b8c7b-b9c2-4d75-a2c3-044d6894b9b9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:02.998258 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfc05a-577f-467e-ad4d-5f4501094945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:02.998850 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfc05a-577f-467e-ad4d-5f4501094945 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:02.999130 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfc05a-577f-467e-ad4d-5f4501094945 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:02.999430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfc05a-577f-467e-ad4d-5f4501094945 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:03.000059 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfc05a-577f-467e-ad4d-5f4501094945 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:25:03.000362 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfc05a-577f-467e-ad4d-5f4501094945 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:03.001716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfc05a-577f-467e-ad4d-5f4501094945 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:03.008568 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 580/1166] 10.222.0.49 () {56 vars in 1115 bytes} [Wed Aug 10 12:25:02 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:25:04.985489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b89cb-0ca8-45f0-b56a-8ac23921cbae None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:04.986038 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b89cb-0ca8-45f0-b56a-8ac23921cbae None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:04.986038 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b89cb-0ca8-45f0-b56a-8ac23921cbae None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:04.987410 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 587/1167] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:25:04.993071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ae7e7-1912-43e7-8969-2eedefa2b861 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:04.993444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ae7e7-1912-43e7-8969-2eedefa2b861 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:04.993625 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ae7e7-1912-43e7-8969-2eedefa2b861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:05.012164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-405ae7e7-1912-43e7-8969-2eedefa2b861 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:25:05.013374 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-405ae7e7-1912-43e7-8969-2eedefa2b861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:05.030117 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:05.078568 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 581/1168] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:25:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:25:05.101502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b89cb-0ca8-45f0-b56a-8ac23921cbae None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:05.208681 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b89cb-0ca8-45f0-b56a-8ac23921cbae None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:05.228830 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:05.242846 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c1491-3d51-4dd1-bb83-f6ba3f6c2c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:05.243405 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c1491-3d51-4dd1-bb83-f6ba3f6c2c59 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:05.243684 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c1491-3d51-4dd1-bb83-f6ba3f6c2c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:05.243962 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c1491-3d51-4dd1-bb83-f6ba3f6c2c59 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:05.244626 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352c1491-3d51-4dd1-bb83-f6ba3f6c2c59 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:05.245982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352c1491-3d51-4dd1-bb83-f6ba3f6c2c59 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:05.325355 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 588/1169] 10.222.0.49 () {60 vars in 1138 bytes} [Wed Aug 10 12:25:05 2022] POST /identity/v3/domains => generated 267 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:25:07.362577 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34249c58-9de9-4a95-8d7e-bd621ee5458e None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:07.363079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34249c58-9de9-4a95-8d7e-bd621ee5458e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:07.363195 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34249c58-9de9-4a95-8d7e-bd621ee5458e None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:07.364476 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 582/1170] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:25:07.370119 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dca813-ab76-492c-807b-47db80069b7d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:07.370535 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dca813-ab76-492c-807b-47db80069b7d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:07.370663 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dca813-ab76-492c-807b-47db80069b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:07.388952 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e2dca813-ab76-492c-807b-47db80069b7d None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:25:07.390118 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e2dca813-ab76-492c-807b-47db80069b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:07.455943 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 589/1171] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:25:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:25:07.480260 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34249c58-9de9-4a95-8d7e-bd621ee5458e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:07.578917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34249c58-9de9-4a95-8d7e-bd621ee5458e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:07.611616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b70191-6ede-4925-a22b-086527d991e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:07.612227 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b70191-6ede-4925-a22b-086527d991e6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:07.612507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b70191-6ede-4925-a22b-086527d991e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:07.612781 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b70191-6ede-4925-a22b-086527d991e6 None admin] PATH_INFO: `/v3/domains/9eec7a49c8574cc38b602d8e21068400` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:07.623114 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b70191-6ede-4925-a22b-086527d991e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=9eec7a49c8574cc38b602d8e21068400)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:07.624560 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b70191-6ede-4925-a22b-086527d991e6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:07.626109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 583/1172] 10.222.0.49 () {56 vars in 1183 bytes} [Wed Aug 10 12:25:07 2022] GET /identity/v3/domains/9eec7a49c8574cc38b602d8e21068400 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:25:07.632109 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dca813-ab76-492c-807b-47db80069b7d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:07.728182 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dca813-ab76-492c-807b-47db80069b7d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:07.738300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:07.761517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381f7057-0e0e-4567-af0a-7a763e599ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:07.762095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f7057-0e0e-4567-af0a-7a763e599ef3 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:07.762374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f7057-0e0e-4567-af0a-7a763e599ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:07.762645 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f7057-0e0e-4567-af0a-7a763e599ef3 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:07.763303 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381f7057-0e0e-4567-af0a-7a763e599ef3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:07.764723 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381f7057-0e0e-4567-af0a-7a763e599ef3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:07.850492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 590/1173] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 10 12:25:07 2022] POST /identity/v3/projects => generated 361 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:25:09.858092 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6990af8-3d3e-417f-92f1-0e3b3e68aeba None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:09.858661 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6990af8-3d3e-417f-92f1-0e3b3e68aeba None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:09.858661 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6990af8-3d3e-417f-92f1-0e3b3e68aeba None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:09.859993 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 584/1174] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:25:09.865047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8a3fa-b866-4a27-bfbc-e681eee1609b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:09.865417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8a3fa-b866-4a27-bfbc-e681eee1609b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:09.865593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8a3fa-b866-4a27-bfbc-e681eee1609b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:09.885155 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-70a8a3fa-b866-4a27-bfbc-e681eee1609b None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:25:09.886326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-70a8a3fa-b866-4a27-bfbc-e681eee1609b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:09.949427 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 591/1175] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:25:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:25:09.973445 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6990af8-3d3e-417f-92f1-0e3b3e68aeba None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:09.989767 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:10.071935 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6990af8-3d3e-417f-92f1-0e3b3e68aeba None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:10.104785 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642d0fd-ffa0-4e08-876d-f0a6e78a7194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:10.105400 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642d0fd-ffa0-4e08-876d-f0a6e78a7194 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:10.105675 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642d0fd-ffa0-4e08-876d-f0a6e78a7194 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:10.105945 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642d0fd-ffa0-4e08-876d-f0a6e78a7194 None admin] PATH_INFO: `/v3/domains/9eec7a49c8574cc38b602d8e21068400` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:10.107035 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5642d0fd-ffa0-4e08-876d-f0a6e78a7194 None admin] RBAC: Authorizing `identity:get_domain(domain_id=9eec7a49c8574cc38b602d8e21068400)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:10.108524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5642d0fd-ffa0-4e08-876d-f0a6e78a7194 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:10.109993 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 585/1176] 10.222.0.49 () {56 vars in 1183 bytes} [Wed Aug 10 12:25:09 2022] GET /identity/v3/domains/9eec7a49c8574cc38b602d8e21068400 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:25:10.115722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8a3fa-b866-4a27-bfbc-e681eee1609b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:10.203791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:10.212676 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8a3fa-b866-4a27-bfbc-e681eee1609b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:10.246541 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54758ba-194c-47ba-bb67-f46033363a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:10.247139 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54758ba-194c-47ba-bb67-f46033363a9a None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:10.247447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54758ba-194c-47ba-bb67-f46033363a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:10.247972 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54758ba-194c-47ba-bb67-f46033363a9a None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:10.248615 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54758ba-194c-47ba-bb67-f46033363a9a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:10.250045 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54758ba-194c-47ba-bb67-f46033363a9a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:10.353593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 592/1177] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 12:25:10 2022] POST /identity/v3/users => generated 311 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:25:12.376428 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac5c57-b739-40b5-82bb-580f8f5a52e2 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:12.376954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac5c57-b739-40b5-82bb-580f8f5a52e2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:12.376954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac5c57-b739-40b5-82bb-580f8f5a52e2 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:12.378252 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 586/1178] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:25:12.383365 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c57e7-1441-44de-8e2a-3961b8b8d098 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:12.383805 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c57e7-1441-44de-8e2a-3961b8b8d098 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:12.383906 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c57e7-1441-44de-8e2a-3961b8b8d098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:12.402096 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-db8c57e7-1441-44de-8e2a-3961b8b8d098 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:25:12.403325 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-db8c57e7-1441-44de-8e2a-3961b8b8d098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:12.468539 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 593/1179] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:25:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:25:12.491480 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac5c57-b739-40b5-82bb-580f8f5a52e2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:12.571758 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:12.590234 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac5c57-b739-40b5-82bb-580f8f5a52e2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:12.623098 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52bbf9-d759-440d-a3b8-5e97373915e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:12.623739 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52bbf9-d759-440d-a3b8-5e97373915e7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:12.624020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52bbf9-d759-440d-a3b8-5e97373915e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:12.624288 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52bbf9-d759-440d-a3b8-5e97373915e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:12.634550 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec52bbf9-d759-440d-a3b8-5e97373915e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:25:12.639165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-ec52bbf9-d759-440d-a3b8-5e97373915e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:25:12.639165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:25:12.639165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:25:12.639165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:25:12.639165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:25:12.639165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:25:12.639165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:25:12.639165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:25:12.639165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:25:12.639165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:25:12.639165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:25:12.639165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:25:12.639165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:25:12.639165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:25:12.639165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:25:12.639165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:25:12.640359 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:25:12.640359 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:25:12.640359 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:25:12.640359 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:25:12.640359 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:25:12.640359 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:25:12.640359 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:25:12.640359 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:25:12.640359 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:25:12.640359 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:25:12.640359 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:25:12.640359 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:25:12.640359 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:25:12.640359 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:25:12.640359 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:25:12.640359 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:25:12.641646 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:25:12.641646 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:25:12.641646 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:25:12.641646 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:25:12.641646 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:25:12.641646 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:25:12.641646 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:25:12.641646 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:25:12.641646 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:25:12.641646 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:25:12.641646 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:25:12.641646 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:25:12.641646 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 587/1180] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:25:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:25:12.646224 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c57e7-1441-44de-8e2a-3961b8b8d098 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:12.721144 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:12.741351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c57e7-1441-44de-8e2a-3961b8b8d098 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:12.775557 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9435005-5b23-4829-920d-29a47f96513c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:12.776151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9435005-5b23-4829-920d-29a47f96513c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:12.776427 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9435005-5b23-4829-920d-29a47f96513c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:12.776696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9435005-5b23-4829-920d-29a47f96513c None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:12.777340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9435005-5b23-4829-920d-29a47f96513c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:25:12.777636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9435005-5b23-4829-920d-29a47f96513c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:12.778973 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9435005-5b23-4829-920d-29a47f96513c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:12.787185 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 594/1181] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:25:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:25:12.793172 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52bbf9-d759-440d-a3b8-5e97373915e7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:12.890586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52bbf9-d759-440d-a3b8-5e97373915e7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:12.923496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389ecf6-5b05-4c20-af6c-7d82af35e9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:12.924105 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389ecf6-5b05-4c20-af6c-7d82af35e9ec None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:12.924384 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389ecf6-5b05-4c20-af6c-7d82af35e9ec None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:12.924658 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389ecf6-5b05-4c20-af6c-7d82af35e9ec None admin] PATH_INFO: `/v3/users/8fc4055631034cd3803b2a7e528128c9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:12.935398 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:12.937682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a389ecf6-5b05-4c20-af6c-7d82af35e9ec None admin] RBAC: Authorizing `identity:get_user(user_id=8fc4055631034cd3803b2a7e528128c9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:12.939130 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a389ecf6-5b05-4c20-af6c-7d82af35e9ec None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:12.940734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 588/1182] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:25:12 2022] GET /identity/v3/users/8fc4055631034cd3803b2a7e528128c9 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:25:12.949669 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9435005-5b23-4829-920d-29a47f96513c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:13.043428 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9435005-5b23-4829-920d-29a47f96513c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:13.076447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c169df9-cd5d-489f-b6db-fd8c39b20d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:13.077039 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c169df9-cd5d-489f-b6db-fd8c39b20d2b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:13.077309 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c169df9-cd5d-489f-b6db-fd8c39b20d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:13.077575 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c169df9-cd5d-489f-b6db-fd8c39b20d2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:13.088018 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:13.142767 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c169df9-cd5d-489f-b6db-fd8c39b20d2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:13.144212 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c169df9-cd5d-489f-b6db-fd8c39b20d2b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:13.238998 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 595/1183] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:25:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:25:13.245021 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389ecf6-5b05-4c20-af6c-7d82af35e9ec None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:13.343673 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389ecf6-5b05-4c20-af6c-7d82af35e9ec None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:13.376829 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d94ce8-ff69-4568-8b85-7a6c4f322c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:13.377452 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d94ce8-ff69-4568-8b85-7a6c4f322c77 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:13.377732 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d94ce8-ff69-4568-8b85-7a6c4f322c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:13.378003 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d94ce8-ff69-4568-8b85-7a6c4f322c77 None admin] PATH_INFO: `/v3/projects/691d4989f8134dc1961b6446c0b23461` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:13.379112 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d94ce8-ff69-4568-8b85-7a6c4f322c77 None admin] RBAC: Authorizing `identity:get_project(project_id=691d4989f8134dc1961b6446c0b23461)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:13.380561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d94ce8-ff69-4568-8b85-7a6c4f322c77 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:13.382172 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 589/1184] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:25:13 2022] GET /identity/v3/projects/691d4989f8134dc1961b6446c0b23461 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:25:13.388444 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c169df9-cd5d-489f-b6db-fd8c39b20d2b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:13.429419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:13.484895 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c169df9-cd5d-489f-b6db-fd8c39b20d2b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:13.517698 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9297f5-0d90-4843-9848-88ee645cfcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:13.518289 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9297f5-0d90-4843-9848-88ee645cfcb4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:13.518565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9297f5-0d90-4843-9848-88ee645cfcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:13.518834 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9297f5-0d90-4843-9848-88ee645cfcb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:13.519567 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9297f5-0d90-4843-9848-88ee645cfcb4 None admin] RBAC: Adding query filter params (role.id=bffbe227cda1418cbb58394ddff6f861, scope.project.id=691d4989f8134dc1961b6446c0b23461, user.id=8fc4055631034cd3803b2a7e528128c9) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:25:13.519876 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9297f5-0d90-4843-9848-88ee645cfcb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:13.521282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9297f5-0d90-4843-9848-88ee645cfcb4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:13.527153 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 596/1185] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:25:13 2022] GET /identity/v3/role_assignments?user.id=8fc4055631034cd3803b2a7e528128c9&scope.project.id=691d4989f8134dc1961b6446c0b23461&role.id=bffbe227cda1418cbb58394ddff6f861 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:25:15.470717 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dad85f-92ed-48ad-95de-5684c449b6a8 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:15.471254 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dad85f-92ed-48ad-95de-5684c449b6a8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:15.471367 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dad85f-92ed-48ad-95de-5684c449b6a8 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:15.472610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 590/1186] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:25:15.478810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c1a73-17a9-4293-a460-d1d9d3126f7b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:15.479248 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c1a73-17a9-4293-a460-d1d9d3126f7b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:15.479371 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c1a73-17a9-4293-a460-d1d9d3126f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:15.496971 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-739c1a73-17a9-4293-a460-d1d9d3126f7b None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:25:15.498402 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-739c1a73-17a9-4293-a460-d1d9d3126f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:15.562168 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 597/1187] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:25:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:25:15.583967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dad85f-92ed-48ad-95de-5684c449b6a8 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:15.650586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:15.683354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dad85f-92ed-48ad-95de-5684c449b6a8 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:15.716555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba1ad5-2040-4d84-80f1-7896d2c272dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:15.717161 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba1ad5-2040-4d84-80f1-7896d2c272dc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:15.717470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba1ad5-2040-4d84-80f1-7896d2c272dc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:15.717741 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba1ad5-2040-4d84-80f1-7896d2c272dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:15.727983 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba1ad5-2040-4d84-80f1-7896d2c272dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:25:15.732549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-8eba1ad5-2040-4d84-80f1-7896d2c272dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:25:15.732549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:25:15.732549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:25:15.732549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:25:15.732549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:25:15.732549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:25:15.732549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:25:15.732549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:25:15.732549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:25:15.732549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:25:15.732549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:25:15.732549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:25:15.732549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:25:15.732549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:25:15.732549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:25:15.732549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:25:15.733606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:25:15.733606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:25:15.733606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:25:15.733606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:25:15.733606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:25:15.733606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:25:15.733606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:25:15.733606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:25:15.733606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:25:15.733606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:25:15.733606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:25:15.733606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:25:15.733606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:25:15.733606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:25:15.733606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:25:15.733606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:25:15.734621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:25:15.734621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:25:15.734621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:25:15.734621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:25:15.734621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:25:15.734621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:25:15.734621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:25:15.734621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:25:15.734621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:25:15.734621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:25:15.734621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:25:15.734621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:25:15.734621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 591/1188] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:25:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:25:15.739564 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c1a73-17a9-4293-a460-d1d9d3126f7b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:15.775375 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:15.839712 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c1a73-17a9-4293-a460-d1d9d3126f7b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:15.871962 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5044d9cd-10de-487d-8a01-8a17ad9d4725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:15.872557 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044d9cd-10de-487d-8a01-8a17ad9d4725 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:15.873098 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044d9cd-10de-487d-8a01-8a17ad9d4725 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:15.873369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044d9cd-10de-487d-8a01-8a17ad9d4725 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:15.874019 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5044d9cd-10de-487d-8a01-8a17ad9d4725 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:25:15.874321 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5044d9cd-10de-487d-8a01-8a17ad9d4725 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:15.875708 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5044d9cd-10de-487d-8a01-8a17ad9d4725 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:15.883969 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 598/1189] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:25:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:25:15.889897 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba1ad5-2040-4d84-80f1-7896d2c272dc None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:15.992821 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba1ad5-2040-4d84-80f1-7896d2c272dc None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:16.018828 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:16.027031 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb710f-07e8-4719-b790-ae15f62ce4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:16.029366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb710f-07e8-4719-b790-ae15f62ce4e3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:16.029366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb710f-07e8-4719-b790-ae15f62ce4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:16.029366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb710f-07e8-4719-b790-ae15f62ce4e3 None admin] PATH_INFO: `/v3/users/8fc4055631034cd3803b2a7e528128c9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:16.029366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb710f-07e8-4719-b790-ae15f62ce4e3 None admin] RBAC: Authorizing `identity:get_user(user_id=8fc4055631034cd3803b2a7e528128c9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:16.030838 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb710f-07e8-4719-b790-ae15f62ce4e3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:16.032443 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 592/1190] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:25:15 2022] GET /identity/v3/users/8fc4055631034cd3803b2a7e528128c9 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:25:16.038236 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5044d9cd-10de-487d-8a01-8a17ad9d4725 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:16.129615 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:16.135803 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5044d9cd-10de-487d-8a01-8a17ad9d4725 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:16.169109 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e83061-6b52-49c2-810f-c1f638fd86bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:16.169725 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e83061-6b52-49c2-810f-c1f638fd86bc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:16.170005 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e83061-6b52-49c2-810f-c1f638fd86bc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:16.170275 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e83061-6b52-49c2-810f-c1f638fd86bc None admin] PATH_INFO: `/v3/projects/691d4989f8134dc1961b6446c0b23461` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:16.171523 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e83061-6b52-49c2-810f-c1f638fd86bc None admin] RBAC: Authorizing `identity:get_project(project_id=691d4989f8134dc1961b6446c0b23461)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:16.172930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e83061-6b52-49c2-810f-c1f638fd86bc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:16.174536 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 599/1191] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:25:16 2022] GET /identity/v3/projects/691d4989f8134dc1961b6446c0b23461 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:25:16.180622 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb710f-07e8-4719-b790-ae15f62ce4e3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:16.277935 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb710f-07e8-4719-b790-ae15f62ce4e3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:16.310582 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c81a1-cb30-49be-92e8-e085bb35365e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:16.311256 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c81a1-cb30-49be-92e8-e085bb35365e None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:16.311545 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c81a1-cb30-49be-92e8-e085bb35365e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:16.311821 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c81a1-cb30-49be-92e8-e085bb35365e None admin] PATH_INFO: `/v3/projects/691d4989f8134dc1961b6446c0b23461/users/8fc4055631034cd3803b2a7e528128c9/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:16.313699 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c81a1-cb30-49be-92e8-e085bb35365e None admin] RBAC: Authorizing `identity:create_grant(project_id=691d4989f8134dc1961b6446c0b23461, user_id=8fc4055631034cd3803b2a7e528128c9, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:16.315143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c81a1-cb30-49be-92e8-e085bb35365e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:16.380989 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 593/1192] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 12:25:16 2022] PUT /identity/v3/projects/691d4989f8134dc1961b6446c0b23461/users/8fc4055631034cd3803b2a7e528128c9/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:25:18.332808 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed407986-0e68-4249-9069-93d6b344f102 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:18.333320 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed407986-0e68-4249-9069-93d6b344f102 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:18.333414 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed407986-0e68-4249-9069-93d6b344f102 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:18.334698 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 600/1193] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:25:18.340378 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4f09d-98d5-4bd4-9d0e-da3356b8cb2d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:18.340798 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4f09d-98d5-4bd4-9d0e-da3356b8cb2d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:18.340928 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4f09d-98d5-4bd4-9d0e-da3356b8cb2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:18.359126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e7c4f09d-98d5-4bd4-9d0e-da3356b8cb2d None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:25:18.360323 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e7c4f09d-98d5-4bd4-9d0e-da3356b8cb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:18.408385 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:18.424324 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 594/1194] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:25:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:25:18.447059 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed407986-0e68-4249-9069-93d6b344f102 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:18.546349 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed407986-0e68-4249-9069-93d6b344f102 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:18.578503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ca348-21cc-463b-98d7-8e283d7b65c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:18.579114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ca348-21cc-463b-98d7-8e283d7b65c7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:18.579422 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ca348-21cc-463b-98d7-8e283d7b65c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:18.579698 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ca348-21cc-463b-98d7-8e283d7b65c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:18.590233 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ca348-21cc-463b-98d7-8e283d7b65c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:25:18.594922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-104ca348-21cc-463b-98d7-8e283d7b65c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:25:18.594922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:25:18.594922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:25:18.594922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:25:18.594922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:25:18.594922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:25:18.594922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:25:18.594922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:25:18.594922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:25:18.594922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:25:18.594922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:25:18.594922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:25:18.594922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:25:18.594922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:25:18.594922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:25:18.594922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:25:18.596042 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:25:18.596042 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:25:18.596042 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:25:18.596042 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:25:18.596042 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:25:18.596042 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:25:18.596042 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:25:18.596042 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:25:18.596042 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:25:18.596042 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:25:18.596042 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:25:18.596042 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:25:18.596042 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:25:18.596042 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:25:18.596042 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:25:18.596042 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:25:18.597225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:25:18.597225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:25:18.597225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:25:18.597225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:25:18.597225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:25:18.597225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:25:18.597225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:25:18.597225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:25:18.597225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:25:18.597225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:25:18.597225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:25:18.597225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:25:18.597225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 601/1195] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:25:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:25:18.602637 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c4f09d-98d5-4bd4-9d0e-da3356b8cb2d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:18.698059 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c4f09d-98d5-4bd4-9d0e-da3356b8cb2d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:18.731371 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd68c5-8644-448e-9026-e5553c5c148a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:18.731958 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd68c5-8644-448e-9026-e5553c5c148a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:18.732236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd68c5-8644-448e-9026-e5553c5c148a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:18.732507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd68c5-8644-448e-9026-e5553c5c148a None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:18.733159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fd68c5-8644-448e-9026-e5553c5c148a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:25:18.733462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fd68c5-8644-448e-9026-e5553c5c148a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:18.734807 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fd68c5-8644-448e-9026-e5553c5c148a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:18.742494 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 595/1196] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:25:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:25:18.748409 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ca348-21cc-463b-98d7-8e283d7b65c7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:18.775103 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:18.847551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ca348-21cc-463b-98d7-8e283d7b65c7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:18.879950 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5070832-641c-4468-92e4-64c2c959a620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:18.880572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5070832-641c-4468-92e4-64c2c959a620 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:18.880852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5070832-641c-4468-92e4-64c2c959a620 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:18.881127 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5070832-641c-4468-92e4-64c2c959a620 None admin] PATH_INFO: `/v3/users/8fc4055631034cd3803b2a7e528128c9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:18.882338 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5070832-641c-4468-92e4-64c2c959a620 None admin] RBAC: Authorizing `identity:get_user(user_id=8fc4055631034cd3803b2a7e528128c9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:18.884529 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5070832-641c-4468-92e4-64c2c959a620 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:18.886125 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 602/1197] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 12:25:18 2022] GET /identity/v3/users/8fc4055631034cd3803b2a7e528128c9 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:25:18.891909 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd68c5-8644-448e-9026-e5553c5c148a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:18.940074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:18.990975 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd68c5-8644-448e-9026-e5553c5c148a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:19.024605 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8789e8c-b1bc-4e20-95f8-8c81f8628be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:19.025206 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8789e8c-b1bc-4e20-95f8-8c81f8628be0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:19.025489 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8789e8c-b1bc-4e20-95f8-8c81f8628be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:19.025764 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8789e8c-b1bc-4e20-95f8-8c81f8628be0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:19.091443 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8789e8c-b1bc-4e20-95f8-8c81f8628be0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:19.092877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8789e8c-b1bc-4e20-95f8-8c81f8628be0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:19.138589 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:19.190619 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 596/1198] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:25:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:25:19.197086 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5070832-641c-4468-92e4-64c2c959a620 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:19.289984 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:19.298331 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5070832-641c-4468-92e4-64c2c959a620 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:19.332325 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21735ed1-70b0-4667-a95e-9832323e8c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:19.332938 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21735ed1-70b0-4667-a95e-9832323e8c38 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:19.333451 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21735ed1-70b0-4667-a95e-9832323e8c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:19.333451 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21735ed1-70b0-4667-a95e-9832323e8c38 None admin] PATH_INFO: `/v3/projects/691d4989f8134dc1961b6446c0b23461` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:19.334975 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21735ed1-70b0-4667-a95e-9832323e8c38 None admin] RBAC: Authorizing `identity:get_project(project_id=691d4989f8134dc1961b6446c0b23461)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:19.337642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21735ed1-70b0-4667-a95e-9832323e8c38 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:19.340234 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 603/1199] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 12:25:19 2022] GET /identity/v3/projects/691d4989f8134dc1961b6446c0b23461 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:25:19.346509 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8789e8c-b1bc-4e20-95f8-8c81f8628be0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:19.442099 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8789e8c-b1bc-4e20-95f8-8c81f8628be0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:19.475011 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582ab7fa-baca-4587-a264-927dabcc0b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:19.475626 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ab7fa-baca-4587-a264-927dabcc0b2f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:19.475904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ab7fa-baca-4587-a264-927dabcc0b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:19.476174 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ab7fa-baca-4587-a264-927dabcc0b2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:19.476904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582ab7fa-baca-4587-a264-927dabcc0b2f None admin] RBAC: Adding query filter params (role.id=bffbe227cda1418cbb58394ddff6f861, scope.project.id=691d4989f8134dc1961b6446c0b23461, user.id=8fc4055631034cd3803b2a7e528128c9) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:25:19.477205 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582ab7fa-baca-4587-a264-927dabcc0b2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:19.478610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582ab7fa-baca-4587-a264-927dabcc0b2f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:19.484188 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 597/1200] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:25:19 2022] GET /identity/v3/role_assignments?user.id=8fc4055631034cd3803b2a7e528128c9&scope.project.id=691d4989f8134dc1961b6446c0b23461&role.id=bffbe227cda1418cbb58394ddff6f861 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:25:29.328318 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=20 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:29.481660 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=94 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:31.448576 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01bf7f9-12c4-4be3-8369-a9bc486079c0 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:31.449696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01bf7f9-12c4-4be3-8369-a9bc486079c0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:31.449696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01bf7f9-12c4-4be3-8369-a9bc486079c0 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:31.450981 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 604/1201] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:26:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:26:31.456158 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e6737-1977-431d-bbb0-5fa3580264ef None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:31.456561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e6737-1977-431d-bbb0-5fa3580264ef None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:31.456756 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e6737-1977-431d-bbb0-5fa3580264ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:31.475371 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a39e6737-1977-431d-bbb0-5fa3580264ef None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:26:31.476542 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a39e6737-1977-431d-bbb0-5fa3580264ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:31.542464 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 598/1202] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:26:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:26:31.566069 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01bf7f9-12c4-4be3-8369-a9bc486079c0 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:31.668547 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01bf7f9-12c4-4be3-8369-a9bc486079c0 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:31.702242 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9133be-bc3d-45ee-a2c4-5be9f5b1390f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:31.702868 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9133be-bc3d-45ee-a2c4-5be9f5b1390f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:31.703150 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9133be-bc3d-45ee-a2c4-5be9f5b1390f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:31.703457 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9133be-bc3d-45ee-a2c4-5be9f5b1390f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:31.713330 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9133be-bc3d-45ee-a2c4-5be9f5b1390f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:31.715358 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9133be-bc3d-45ee-a2c4-5be9f5b1390f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:31.729642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-1c9133be-bc3d-45ee-a2c4-5be9f5b1390f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:26:31.729642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:31.729642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:31.729642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:31.729642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:31.729642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:31.729642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:31.729642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:31.729642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:31.729642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:31.729642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:31.729642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:31.729642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:26:31.729642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:26:31.729642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:26:31.729642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:26:31.734053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:31.734053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:31.734053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:31.734053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:31.734053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:31.734053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:31.734053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:31.734053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:31.734053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:31.734053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:31.734053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:31.734053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:31.734053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:31.734053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:31.734053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:31.734053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:31.734053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:26:31.735297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:26:31.735297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:26:31.735297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:26:31.735297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 605/1203] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:26:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:31.737036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e6737-1977-431d-bbb0-5fa3580264ef None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:31.837104 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e6737-1977-431d-bbb0-5fa3580264ef None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:31.865180 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:31.872635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ac4eb-64c0-46ee-bd3c-a575a15d06a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:31.876032 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ac4eb-64c0-46ee-bd3c-a575a15d06a4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:31.876032 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ac4eb-64c0-46ee-bd3c-a575a15d06a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:31.876032 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ac4eb-64c0-46ee-bd3c-a575a15d06a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:31.876032 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ac4eb-64c0-46ee-bd3c-a575a15d06a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:31.876032 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ac4eb-64c0-46ee-bd3c-a575a15d06a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:31.876454 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ac4eb-64c0-46ee-bd3c-a575a15d06a4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:31.887607 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 599/1204] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:31.893276 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9133be-bc3d-45ee-a2c4-5be9f5b1390f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:31.946972 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:31.993556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9133be-bc3d-45ee-a2c4-5be9f5b1390f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:32.027094 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102163a-d870-41f5-bab7-fae9fd8c9eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:32.028244 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102163a-d870-41f5-bab7-fae9fd8c9eaa None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:32.028753 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102163a-d870-41f5-bab7-fae9fd8c9eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:32.029190 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102163a-d870-41f5-bab7-fae9fd8c9eaa None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:32.030350 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d102163a-d870-41f5-bab7-fae9fd8c9eaa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:32.031942 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d102163a-d870-41f5-bab7-fae9fd8c9eaa None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:32.108442 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 606/1205] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 12:26:31 2022] POST /identity/v3/users => generated 251 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:26:34.128089 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55843c6-cfa0-4b73-a75e-808bac18d474 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:34.128640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55843c6-cfa0-4b73-a75e-808bac18d474 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:34.128640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55843c6-cfa0-4b73-a75e-808bac18d474 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:34.130032 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 600/1206] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:26:34.135074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7b58e-50f3-4ff5-a3bf-0b53fea33eb8 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:34.135566 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7b58e-50f3-4ff5-a3bf-0b53fea33eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:34.135664 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7b58e-50f3-4ff5-a3bf-0b53fea33eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:34.154274 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7be7b58e-50f3-4ff5-a3bf-0b53fea33eb8 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:26:34.155487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7be7b58e-50f3-4ff5-a3bf-0b53fea33eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:34.220741 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 607/1207] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:26:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:26:34.243722 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55843c6-cfa0-4b73-a75e-808bac18d474 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:34.348102 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55843c6-cfa0-4b73-a75e-808bac18d474 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:34.382764 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf134570-d626-4e6e-aaa3-1628461597f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:34.383345 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf134570-d626-4e6e-aaa3-1628461597f2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:34.383622 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf134570-d626-4e6e-aaa3-1628461597f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:34.383891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf134570-d626-4e6e-aaa3-1628461597f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:34.394760 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf134570-d626-4e6e-aaa3-1628461597f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:26:34.399438 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-bf134570-d626-4e6e-aaa3-1628461597f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:26:34.399438 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:34.399438 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:34.399438 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:34.399438 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:34.399438 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:34.399438 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:34.399438 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:34.399438 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:34.399438 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:34.399438 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:34.399438 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:34.399438 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:26:34.399438 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:26:34.399438 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:26:34.399438 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:26:34.400549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:26:34.400549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:26:34.400549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:26:34.400549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:26:34.400549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:34.400549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:34.400549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:34.400549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:34.400549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:34.400549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:34.400549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:34.400549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:34.400549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:34.400549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:34.400549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:34.400549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:34.401784 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:34.401784 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:34.401784 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:34.401784 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:34.401784 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:26:34.401784 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:26:34.401784 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:26:34.401784 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:26:34.401784 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:26:34.401784 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:26:34.401784 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:26:34.401784 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:26:34.401784 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 601/1208] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 12:26:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:34.406912 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7b58e-50f3-4ff5-a3bf-0b53fea33eb8 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:34.447592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:34.506387 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7b58e-50f3-4ff5-a3bf-0b53fea33eb8 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:34.538532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c833f7-7162-445f-b9ad-758122cc45a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:34.539128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c833f7-7162-445f-b9ad-758122cc45a3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:34.539433 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c833f7-7162-445f-b9ad-758122cc45a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:34.539699 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c833f7-7162-445f-b9ad-758122cc45a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:34.540345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c833f7-7162-445f-b9ad-758122cc45a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:34.540642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c833f7-7162-445f-b9ad-758122cc45a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:34.542051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c833f7-7162-445f-b9ad-758122cc45a3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:34.550117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 608/1209] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 12:26:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:34.555964 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf134570-d626-4e6e-aaa3-1628461597f2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:34.607232 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:34.656249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf134570-d626-4e6e-aaa3-1628461597f2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:34.688452 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5527de6-d844-46f7-a7e5-be462c51a6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:34.689002 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5527de6-d844-46f7-a7e5-be462c51a6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:34.689277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5527de6-d844-46f7-a7e5-be462c51a6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:34.689547 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5527de6-d844-46f7-a7e5-be462c51a6e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:34.700863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5527de6-d844-46f7-a7e5-be462c51a6e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:34.702323 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5527de6-d844-46f7-a7e5-be462c51a6e3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:34.716748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a5527de6-d844-46f7-a7e5-be462c51a6e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:26:34.716748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:34.716748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:34.716748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:34.716748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:34.716748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:34.716748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:34.716748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:34.716748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:34.716748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:34.716748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:34.716748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:34.716748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:26:34.716748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:26:34.716748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:26:34.716748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:26:34.717872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:34.717872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:34.717872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:34.717872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:34.717872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:34.717872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:34.717872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:34.717872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:34.717872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:34.717872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:34.717872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:34.717872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:34.717872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:34.717872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:34.717872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:34.717872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:34.717872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:26:34.718932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:26:34.718932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:26:34.718932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:26:34.718932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 602/1210] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:26:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:34.723659 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c833f7-7162-445f-b9ad-758122cc45a3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:34.819268 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:34.821953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c833f7-7162-445f-b9ad-758122cc45a3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:34.854401 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6bf82-0c04-4077-b58c-6f710b7a54db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:34.855031 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6bf82-0c04-4077-b58c-6f710b7a54db None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:34.855370 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6bf82-0c04-4077-b58c-6f710b7a54db None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:34.855639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6bf82-0c04-4077-b58c-6f710b7a54db None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:34.856305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6bf82-0c04-4077-b58c-6f710b7a54db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:34.856605 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6bf82-0c04-4077-b58c-6f710b7a54db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:34.857959 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6bf82-0c04-4077-b58c-6f710b7a54db None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:34.868949 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 609/1211] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:26:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:34.875486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5527de6-d844-46f7-a7e5-be462c51a6e3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:34.973922 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5527de6-d844-46f7-a7e5-be462c51a6e3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:34.975387 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:35.007687 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dfc065-8744-4703-af09-8a3415f25088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:35.008305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfc065-8744-4703-af09-8a3415f25088 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:35.008586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfc065-8744-4703-af09-8a3415f25088 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:35.008855 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfc065-8744-4703-af09-8a3415f25088 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:35.016493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dfc065-8744-4703-af09-8a3415f25088 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:35.017900 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dfc065-8744-4703-af09-8a3415f25088 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:35.029051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-61dfc065-8744-4703-af09-8a3415f25088 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 12:26:35.029051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:35.029051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:35.029051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:35.029051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:35.029051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:35.029051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:35.029051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:35.029051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:35.029051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:35.029051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:35.029051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:35.029051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:26:35.029051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:26:35.029051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:26:35.029051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:26:35.030706 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:35.030706 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:35.030706 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:26:35.030706 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:26:35.030706 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:26:35.030706 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:26:35.030706 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:35.030706 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:35.030706 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:35.030706 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:35.030706 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:35.030706 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:35.030706 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:35.030706 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:35.030706 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:35.030706 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:35.030706 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:35.030706 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:35.031908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:35.031908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:35.031908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:26:35.031908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:26:35.031908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:26:35.031908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:26:35.031908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:26:35.031908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:26:35.031908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 12:26:35.031908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:26:35.031908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 603/1212] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:26:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:35.035976 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6bf82-0c04-4077-b58c-6f710b7a54db None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:35.134242 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6bf82-0c04-4077-b58c-6f710b7a54db None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:35.166423 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d39498-4cfc-4366-bbe4-9c2fcd861709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:35.167027 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d39498-4cfc-4366-bbe4-9c2fcd861709 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:35.167336 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d39498-4cfc-4366-bbe4-9c2fcd861709 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:35.167611 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d39498-4cfc-4366-bbe4-9c2fcd861709 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:35.168905 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d39498-4cfc-4366-bbe4-9c2fcd861709 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:35.169272 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d39498-4cfc-4366-bbe4-9c2fcd861709 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:35.170743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d39498-4cfc-4366-bbe4-9c2fcd861709 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:35.188443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 610/1213] 10.222.0.49 () {56 vars in 1143 bytes} [Wed Aug 10 12:26:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:35.194288 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dfc065-8744-4703-af09-8a3415f25088 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:35.297120 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dfc065-8744-4703-af09-8a3415f25088 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:35.330352 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d5535-ef18-47cd-846d-70a63265564c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:35.330949 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d5535-ef18-47cd-846d-70a63265564c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:35.331256 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d5535-ef18-47cd-846d-70a63265564c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:35.331528 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d5535-ef18-47cd-846d-70a63265564c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:35.360959 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:35.399198 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d5535-ef18-47cd-846d-70a63265564c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:35.400680 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d5535-ef18-47cd-846d-70a63265564c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:35.495234 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 604/1214] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:26:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:26:35.502298 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d39498-4cfc-4366-bbe4-9c2fcd861709 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:35.520991 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:35.602027 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d39498-4cfc-4366-bbe4-9c2fcd861709 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:35.635566 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c173f7d-ca67-46ff-bf26-6eabb27c6fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:35.636225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c173f7d-ca67-46ff-bf26-6eabb27c6fab None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:35.636557 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c173f7d-ca67-46ff-bf26-6eabb27c6fab None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:35.636828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c173f7d-ca67-46ff-bf26-6eabb27c6fab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:35.648054 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c173f7d-ca67-46ff-bf26-6eabb27c6fab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:35.649614 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c173f7d-ca67-46ff-bf26-6eabb27c6fab None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:35.664606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-0c173f7d-ca67-46ff-bf26-6eabb27c6fab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:26:35.664606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:35.664606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:35.664606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:35.664606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:35.664606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:35.664606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:35.664606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:35.664606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:35.664606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:35.664606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:35.664606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:35.664606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:26:35.664606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:26:35.664606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:26:35.664606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:26:35.665977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:35.665977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:35.665977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:35.665977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:35.665977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:35.665977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:35.665977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:35.665977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:35.665977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:35.665977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:35.665977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:35.665977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:35.665977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:35.665977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:35.665977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:35.665977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:35.665977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:26:35.667253 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:26:35.667253 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:26:35.667253 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:26:35.667253 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 611/1215] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:26:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:35.672191 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d5535-ef18-47cd-846d-70a63265564c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:35.733227 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:35.770182 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d5535-ef18-47cd-846d-70a63265564c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:35.805021 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa22ed-32e9-4d08-a360-876e20fc0f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:35.805605 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa22ed-32e9-4d08-a360-876e20fc0f91 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:35.805882 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa22ed-32e9-4d08-a360-876e20fc0f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:35.806153 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa22ed-32e9-4d08-a360-876e20fc0f91 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:35.806780 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa22ed-32e9-4d08-a360-876e20fc0f91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:35.807080 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa22ed-32e9-4d08-a360-876e20fc0f91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:35.808505 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa22ed-32e9-4d08-a360-876e20fc0f91 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:35.820283 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 605/1216] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:26:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:35.826383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c173f7d-ca67-46ff-bf26-6eabb27c6fab None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:35.896095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:35.927668 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c173f7d-ca67-46ff-bf26-6eabb27c6fab None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:35.961471 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5f209-d2dc-4c0e-8aec-f16dbaf3ea75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:35.962151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5f209-d2dc-4c0e-8aec-f16dbaf3ea75 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:35.962429 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5f209-d2dc-4c0e-8aec-f16dbaf3ea75 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:35.962702 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5f209-d2dc-4c0e-8aec-f16dbaf3ea75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:35.970147 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5f209-d2dc-4c0e-8aec-f16dbaf3ea75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:35.971610 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5f209-d2dc-4c0e-8aec-f16dbaf3ea75 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:35.983207 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-0fd5f209-d2dc-4c0e-8aec-f16dbaf3ea75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:26:35.983207 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:35.983207 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:35.983207 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:35.983207 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:35.983207 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:35.983207 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:35.983207 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:35.983207 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:35.983207 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:35.983207 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:35.983207 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:35.983207 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:26:35.983207 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:26:35.983207 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:26:35.984287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:26:35.984287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:35.984287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:35.984287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:35.984287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:35.984287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:35.984287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:35.984287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:35.984287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:35.984287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:35.984287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:35.984287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:35.984287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:35.984287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:35.984287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:35.984287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:35.984287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:35.985240 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:26:35.985240 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:26:35.985240 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:26:35.985240 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:26:35.985240 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:26:35.985240 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:26:35.985240 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:26:35.985240 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:26:35.985240 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 612/1217] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:26:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:35.990204 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa22ed-32e9-4d08-a360-876e20fc0f91 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:36.092290 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa22ed-32e9-4d08-a360-876e20fc0f91 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:36.114343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:36.126005 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978386b-6292-490b-89bb-ae4be188e3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:36.126608 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978386b-6292-490b-89bb-ae4be188e3a9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:36.126886 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978386b-6292-490b-89bb-ae4be188e3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:36.142012 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978386b-6292-490b-89bb-ae4be188e3a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:36.142012 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978386b-6292-490b-89bb-ae4be188e3a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:36.142012 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978386b-6292-490b-89bb-ae4be188e3a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:36.142012 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978386b-6292-490b-89bb-ae4be188e3a9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:36.142012 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 606/1218] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:26:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:36.147530 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5f209-d2dc-4c0e-8aec-f16dbaf3ea75 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:36.245557 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5f209-d2dc-4c0e-8aec-f16dbaf3ea75 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:36.275325 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:36.281151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e36f45-fda5-4724-8e4e-4c9bf55f6bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:36.281759 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e36f45-fda5-4724-8e4e-4c9bf55f6bcd None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:36.282038 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e36f45-fda5-4724-8e4e-4c9bf55f6bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:36.282309 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e36f45-fda5-4724-8e4e-4c9bf55f6bcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:36.283023 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e36f45-fda5-4724-8e4e-4c9bf55f6bcd None admin] RBAC: Adding query filter params (role.id=a9bbb011afca4176a3def9356cb5f165, scope.project.id=8e201e5f281a4977be503e22daf149d7, user.id=3ff152e22374407eb72caf648a763107) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:36.283373 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e36f45-fda5-4724-8e4e-4c9bf55f6bcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:36.284777 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e36f45-fda5-4724-8e4e-4c9bf55f6bcd None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:36.290094 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 613/1219] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:26:36 2022] GET /identity/v3/role_assignments?user.id=3ff152e22374407eb72caf648a763107&scope.project.id=8e201e5f281a4977be503e22daf149d7&role.id=a9bbb011afca4176a3def9356cb5f165 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:38.257051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbf7e3-c885-4aad-98a2-a68e822fd779 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:38.257581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbf7e3-c885-4aad-98a2-a68e822fd779 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:38.257581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbf7e3-c885-4aad-98a2-a68e822fd779 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:38.258944 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 607/1220] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:26:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:26:38.264890 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3db55-1c5e-48d4-9217-1c4eb929777f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:38.265327 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3db55-1c5e-48d4-9217-1c4eb929777f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:38.265466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3db55-1c5e-48d4-9217-1c4eb929777f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:38.284069 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a9d3db55-1c5e-48d4-9217-1c4eb929777f None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:26:38.285406 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a9d3db55-1c5e-48d4-9217-1c4eb929777f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:38.350836 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 614/1221] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:26:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:26:38.372735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfbf7e3-c885-4aad-98a2-a68e822fd779 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:38.475120 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfbf7e3-c885-4aad-98a2-a68e822fd779 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:38.508774 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3db52f-f5df-4bd6-bbd6-9b2f07ba2ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:38.509337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3db52f-f5df-4bd6-bbd6-9b2f07ba2ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:38.509612 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3db52f-f5df-4bd6-bbd6-9b2f07ba2ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:38.509882 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3db52f-f5df-4bd6-bbd6-9b2f07ba2ce8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:38.520539 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3db52f-f5df-4bd6-bbd6-9b2f07ba2ce8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:26:38.525453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-7b3db52f-f5df-4bd6-bbd6-9b2f07ba2ce8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:26:38.525453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:38.525453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:38.525453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:38.525453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:38.525453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:38.525453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:38.525453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:38.525453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:38.525453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:38.525453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:38.525453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:38.525453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:26:38.525453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:26:38.525453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:26:38.525453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:26:38.526557 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:26:38.526557 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:26:38.526557 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:26:38.526557 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:26:38.526557 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:38.526557 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:38.526557 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:38.526557 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:38.526557 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:38.526557 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:38.526557 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:38.526557 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:38.526557 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:38.526557 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:38.526557 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:38.526557 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:38.527463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:38.527463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:38.527463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:38.527463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:38.527463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:26:38.527463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:26:38.527463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:26:38.527463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:26:38.527463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:26:38.527463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:26:38.527463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:26:38.527463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:26:38.527463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 608/1222] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 12:26:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:38.533530 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3db55-1c5e-48d4-9217-1c4eb929777f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:38.631775 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3db55-1c5e-48d4-9217-1c4eb929777f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:38.655078 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:38.665313 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a9198-ede9-42f5-9ce5-55691fedbfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:38.665976 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a9198-ede9-42f5-9ce5-55691fedbfd6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:38.666257 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a9198-ede9-42f5-9ce5-55691fedbfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:38.666529 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a9198-ede9-42f5-9ce5-55691fedbfd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:38.667205 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a9198-ede9-42f5-9ce5-55691fedbfd6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:38.667506 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a9198-ede9-42f5-9ce5-55691fedbfd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:38.671713 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a9198-ede9-42f5-9ce5-55691fedbfd6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:38.677297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 615/1223] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 12:26:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:38.683257 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3db52f-f5df-4bd6-bbd6-9b2f07ba2ce8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:38.728710 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:38.785394 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3db52f-f5df-4bd6-bbd6-9b2f07ba2ce8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:38.818618 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165fefb-3f19-4e51-ba69-6c1d35909482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:38.819270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165fefb-3f19-4e51-ba69-6c1d35909482 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:38.819584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165fefb-3f19-4e51-ba69-6c1d35909482 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:38.819874 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165fefb-3f19-4e51-ba69-6c1d35909482 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:38.830666 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3165fefb-3f19-4e51-ba69-6c1d35909482 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:38.832255 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3165fefb-3f19-4e51-ba69-6c1d35909482 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:38.847844 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-3165fefb-3f19-4e51-ba69-6c1d35909482 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:26:38.847844 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:38.847844 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:38.847844 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:38.847844 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:38.847844 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:38.847844 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:38.847844 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:38.847844 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:38.847844 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:38.847844 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:38.847844 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:38.847844 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:26:38.847844 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:26:38.847844 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:26:38.847844 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:26:38.848940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:38.848940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:38.848940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:38.848940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:38.848940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:38.848940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:38.848940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:38.848940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:38.848940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:38.848940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:38.848940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:38.848940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:38.848940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:38.848940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:38.848940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:38.848940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:38.848940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:26:38.849999 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:26:38.849999 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:26:38.849999 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:26:38.849999 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 609/1224] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:26:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:38.854748 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a9198-ede9-42f5-9ce5-55691fedbfd6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:38.954465 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a9198-ede9-42f5-9ce5-55691fedbfd6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:38.987850 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43b098-5002-4fd9-82e2-7db0a1b69c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:38.988448 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43b098-5002-4fd9-82e2-7db0a1b69c8e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:38.988731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43b098-5002-4fd9-82e2-7db0a1b69c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:38.989008 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43b098-5002-4fd9-82e2-7db0a1b69c8e None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:38.989641 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43b098-5002-4fd9-82e2-7db0a1b69c8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:38.989948 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43b098-5002-4fd9-82e2-7db0a1b69c8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:38.991336 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43b098-5002-4fd9-82e2-7db0a1b69c8e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:39.003093 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 616/1225] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:39.008855 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165fefb-3f19-4e51-ba69-6c1d35909482 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:39.103496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:39.110007 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165fefb-3f19-4e51-ba69-6c1d35909482 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:39.142995 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f031ef3-749e-4539-b3fa-59807a6fd89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:39.143660 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f031ef3-749e-4539-b3fa-59807a6fd89a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:39.143941 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f031ef3-749e-4539-b3fa-59807a6fd89a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:39.144213 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f031ef3-749e-4539-b3fa-59807a6fd89a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:39.150875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f031ef3-749e-4539-b3fa-59807a6fd89a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:39.152341 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f031ef3-749e-4539-b3fa-59807a6fd89a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:39.164045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-3f031ef3-749e-4539-b3fa-59807a6fd89a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 12:26:39.164045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:39.164045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:39.164045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:39.164045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:39.164045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:39.164045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:39.164045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:39.164045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:39.164045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:39.164045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:39.164045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:39.164045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:26:39.164045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:26:39.164045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:26:39.164045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:26:39.165310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:39.165310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:39.165310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:26:39.165310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:26:39.165310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:26:39.165310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:26:39.165310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:39.165310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:39.165310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:39.165310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:39.165310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:39.165310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:39.165310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:39.165310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:39.165310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:39.165310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:39.165310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:39.165310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:39.166493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:39.166493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:39.166493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:26:39.166493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:26:39.166493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:26:39.166493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:26:39.166493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:26:39.166493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:26:39.166493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 12:26:39.166493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:26:39.166493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 610/1226] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:26:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:39.171134 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43b098-5002-4fd9-82e2-7db0a1b69c8e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:39.217728 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:39.270964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43b098-5002-4fd9-82e2-7db0a1b69c8e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:39.305727 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cecbfec-0a1e-446a-8f4c-1ff4519d24bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:39.306329 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecbfec-0a1e-446a-8f4c-1ff4519d24bc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:39.306610 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecbfec-0a1e-446a-8f4c-1ff4519d24bc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:39.306881 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecbfec-0a1e-446a-8f4c-1ff4519d24bc None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:39.307625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cecbfec-0a1e-446a-8f4c-1ff4519d24bc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:39.307931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cecbfec-0a1e-446a-8f4c-1ff4519d24bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:39.309844 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cecbfec-0a1e-446a-8f4c-1ff4519d24bc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:39.326815 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 617/1227] 10.222.0.49 () {56 vars in 1143 bytes} [Wed Aug 10 12:26:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:39.332696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f031ef3-749e-4539-b3fa-59807a6fd89a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:39.432746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f031ef3-749e-4539-b3fa-59807a6fd89a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:39.466614 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d576a-c8c1-4165-9446-20cfa1abe544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:39.467265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d576a-c8c1-4165-9446-20cfa1abe544 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:39.467545 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d576a-c8c1-4165-9446-20cfa1abe544 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:39.467817 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d576a-c8c1-4165-9446-20cfa1abe544 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:39.478153 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d576a-c8c1-4165-9446-20cfa1abe544 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:39.479629 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d576a-c8c1-4165-9446-20cfa1abe544 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:39.494463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-9a3d576a-c8c1-4165-9446-20cfa1abe544 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:26:39.494463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:39.494463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:39.494463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:39.494463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:39.494463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:39.494463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:39.494463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:39.494463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:39.494463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:39.494463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:39.494463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:39.494463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:26:39.494463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:26:39.494463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:26:39.494463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:26:39.495594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:39.495594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:39.495594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:39.495594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:39.495594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:39.495594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:39.495594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:39.495594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:39.495594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:39.495594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:39.495594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:39.495594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:39.495594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:39.495594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:39.495594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:39.495594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:39.495594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:26:39.497209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:26:39.497209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:26:39.497209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:26:39.497209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 611/1228] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:26:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:39.501942 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cecbfec-0a1e-446a-8f4c-1ff4519d24bc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:39.603527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cecbfec-0a1e-446a-8f4c-1ff4519d24bc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:39.605998 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:39.637966 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01f0683-c1c5-4997-a1b9-fdc765ef45ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:39.638608 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f0683-c1c5-4997-a1b9-fdc765ef45ce None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:39.638983 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f0683-c1c5-4997-a1b9-fdc765ef45ce None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:39.639227 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f0683-c1c5-4997-a1b9-fdc765ef45ce None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:39.639833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01f0683-c1c5-4997-a1b9-fdc765ef45ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:39.640132 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01f0683-c1c5-4997-a1b9-fdc765ef45ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:39.641472 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01f0683-c1c5-4997-a1b9-fdc765ef45ce None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:39.652482 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 618/1229] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:39.658920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d576a-c8c1-4165-9446-20cfa1abe544 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:39.668656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:39.762733 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d576a-c8c1-4165-9446-20cfa1abe544 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:39.797330 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455da296-0211-4e8d-9778-2ad1dc4af577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:39.797949 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455da296-0211-4e8d-9778-2ad1dc4af577 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:39.798223 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455da296-0211-4e8d-9778-2ad1dc4af577 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:39.798490 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455da296-0211-4e8d-9778-2ad1dc4af577 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:39.805100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455da296-0211-4e8d-9778-2ad1dc4af577 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:39.806526 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455da296-0211-4e8d-9778-2ad1dc4af577 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:39.817999 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-455da296-0211-4e8d-9778-2ad1dc4af577 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:26:39.817999 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:39.817999 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:39.817999 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:39.817999 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:39.817999 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:39.817999 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:39.817999 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:39.817999 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:39.817999 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:39.817999 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:39.817999 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:39.817999 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:26:39.817999 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:26:39.817999 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:26:39.819716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:26:39.819716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:39.819716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:39.819716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:39.819716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:39.819716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:39.819716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:39.819716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:39.819716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:39.819716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:39.819716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:39.819716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:39.819716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:39.819716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:39.819716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:39.819716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:39.819716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:39.820730 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:26:39.820730 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:26:39.820730 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:26:39.820730 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:26:39.820730 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:26:39.820730 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:26:39.820730 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:26:39.820730 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:26:39.820730 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 612/1230] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:26:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:39.825873 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01f0683-c1c5-4997-a1b9-fdc765ef45ce None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:39.926496 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01f0683-c1c5-4997-a1b9-fdc765ef45ce None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:39.959391 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ddd2e-8ac4-4201-b496-d946fe480363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:39.959994 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ddd2e-8ac4-4201-b496-d946fe480363 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:39.960272 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ddd2e-8ac4-4201-b496-d946fe480363 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:39.960545 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ddd2e-8ac4-4201-b496-d946fe480363 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:39.961223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ddd2e-8ac4-4201-b496-d946fe480363 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:39.961526 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ddd2e-8ac4-4201-b496-d946fe480363 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:39.962943 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ddd2e-8ac4-4201-b496-d946fe480363 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:39.975074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 619/1231] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:26:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:39.981270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455da296-0211-4e8d-9778-2ad1dc4af577 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:40.045269 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:40.079072 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455da296-0211-4e8d-9778-2ad1dc4af577 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:40.113080 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a23891-a938-4f2e-9b85-4e67065ebefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:40.113756 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a23891-a938-4f2e-9b85-4e67065ebefb None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:40.114034 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a23891-a938-4f2e-9b85-4e67065ebefb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:40.114334 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a23891-a938-4f2e-9b85-4e67065ebefb None admin] PATH_INFO: `/v3/projects/8e201e5f281a4977be503e22daf149d7/users/3ff152e22374407eb72caf648a763107/roles/a9bbb011afca4176a3def9356cb5f165` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:40.129779 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a23891-a938-4f2e-9b85-4e67065ebefb None admin] RBAC: Authorizing `identity:create_grant(project_id=8e201e5f281a4977be503e22daf149d7, user_id=3ff152e22374407eb72caf648a763107, role_id=a9bbb011afca4176a3def9356cb5f165)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:40.131316 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a23891-a938-4f2e-9b85-4e67065ebefb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:40.206113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 613/1232] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 12:26:39 2022] PUT /identity/v3/projects/8e201e5f281a4977be503e22daf149d7/users/3ff152e22374407eb72caf648a763107/roles/a9bbb011afca4176a3def9356cb5f165 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:26:42.179652 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf9a82-c60e-4f8b-9461-0e94e640af14 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:42.180223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf9a82-c60e-4f8b-9461-0e94e640af14 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:42.180223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf9a82-c60e-4f8b-9461-0e94e640af14 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:42.181533 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 620/1233] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:26:42.186988 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02cb70-6c36-4265-bb84-0333b9ec2dbb None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:42.187440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02cb70-6c36-4265-bb84-0333b9ec2dbb None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:42.190803 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02cb70-6c36-4265-bb84-0333b9ec2dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:42.206042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7a02cb70-6c36-4265-bb84-0333b9ec2dbb None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:26:42.207275 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7a02cb70-6c36-4265-bb84-0333b9ec2dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:42.277073 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 614/1234] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:26:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:26:42.300263 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf9a82-c60e-4f8b-9461-0e94e640af14 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:42.326720 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:42.400851 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf9a82-c60e-4f8b-9461-0e94e640af14 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:42.433505 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd383f-68b2-4cf9-ad26-ad471957e2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:42.434584 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd383f-68b2-4cf9-ad26-ad471957e2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:42.434876 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd383f-68b2-4cf9-ad26-ad471957e2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:42.435148 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd383f-68b2-4cf9-ad26-ad471957e2a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:42.445904 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd383f-68b2-4cf9-ad26-ad471957e2a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:26:42.451351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-aedd383f-68b2-4cf9-ad26-ad471957e2a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:26:42.451351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:42.451351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:42.451351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:42.451351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:42.451351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:42.451351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:42.451351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:42.451351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:42.451351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:42.451351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:42.451351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:42.451351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:26:42.451351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:26:42.451351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:26:42.451351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:26:42.452481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:26:42.452481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:26:42.452481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:26:42.452481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:26:42.452481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:42.452481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:42.452481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:42.452481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:42.452481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:42.452481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:42.452481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:42.452481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:42.452481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:42.452481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:42.452481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:42.452481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:42.453596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:42.453596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:42.453596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:42.453596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:42.453596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:26:42.453596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:26:42.453596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:26:42.453596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:26:42.453596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:26:42.453596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:26:42.453596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:26:42.453596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:26:42.453596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 621/1235] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 12:26:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:42.458515 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02cb70-6c36-4265-bb84-0333b9ec2dbb None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:42.504402 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:42.556565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02cb70-6c36-4265-bb84-0333b9ec2dbb None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:42.591895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9c726-8beb-4837-91c3-cbc14ef493af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:42.592554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9c726-8beb-4837-91c3-cbc14ef493af None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:42.592836 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9c726-8beb-4837-91c3-cbc14ef493af None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:42.593103 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9c726-8beb-4837-91c3-cbc14ef493af None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:42.593750 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e9c726-8beb-4837-91c3-cbc14ef493af None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:42.594085 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e9c726-8beb-4837-91c3-cbc14ef493af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:42.595458 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e9c726-8beb-4837-91c3-cbc14ef493af None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:42.603161 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 615/1236] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 12:26:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:42.609474 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd383f-68b2-4cf9-ad26-ad471957e2a4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:42.692774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:42.708166 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd383f-68b2-4cf9-ad26-ad471957e2a4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:42.741605 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5500512-2435-4437-8cb5-17b64964f3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:42.742224 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5500512-2435-4437-8cb5-17b64964f3dd None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:42.742502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5500512-2435-4437-8cb5-17b64964f3dd None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:42.742775 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5500512-2435-4437-8cb5-17b64964f3dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:42.753269 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5500512-2435-4437-8cb5-17b64964f3dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:42.754689 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5500512-2435-4437-8cb5-17b64964f3dd None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:42.769755 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-d5500512-2435-4437-8cb5-17b64964f3dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:26:42.769755 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:42.769755 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:42.769755 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:42.769755 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:42.769755 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:42.769755 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:42.769755 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:42.769755 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:42.769755 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:42.769755 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:42.769755 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:42.769755 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:26:42.769755 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:26:42.769755 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:26:42.769755 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:26:42.775372 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:42.775372 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:42.775372 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:42.775372 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:42.775372 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:42.775372 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:42.775372 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:42.775372 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:42.775372 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:42.775372 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:42.775372 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:42.775372 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:42.775372 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:42.775372 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:42.775372 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:42.775372 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:42.775372 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:26:42.776402 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:26:42.776402 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:26:42.776402 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:26:42.776402 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 622/1237] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:26:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:42.776840 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9c726-8beb-4837-91c3-cbc14ef493af None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:42.876052 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9c726-8beb-4837-91c3-cbc14ef493af None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:42.884593 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:42.908298 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc56ac-316d-4047-ac87-ce07779f3dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:42.908897 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc56ac-316d-4047-ac87-ce07779f3dcb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:42.909176 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc56ac-316d-4047-ac87-ce07779f3dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:42.909447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc56ac-316d-4047-ac87-ce07779f3dcb None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:42.910074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc56ac-316d-4047-ac87-ce07779f3dcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:42.910378 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc56ac-316d-4047-ac87-ce07779f3dcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:42.911775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc56ac-316d-4047-ac87-ce07779f3dcb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:42.924879 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 616/1238] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:26:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:42.930747 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5500512-2435-4437-8cb5-17b64964f3dd None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:43.030857 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5500512-2435-4437-8cb5-17b64964f3dd None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:43.063845 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db41a1d-6972-4965-86a2-14b67c819057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:43.064464 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db41a1d-6972-4965-86a2-14b67c819057 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:43.064739 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db41a1d-6972-4965-86a2-14b67c819057 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:43.065006 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db41a1d-6972-4965-86a2-14b67c819057 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:43.067604 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:43.072499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db41a1d-6972-4965-86a2-14b67c819057 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:43.074128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db41a1d-6972-4965-86a2-14b67c819057 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:43.084897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-0db41a1d-6972-4965-86a2-14b67c819057 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 12:26:43.084897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:43.084897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:43.084897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:43.084897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:43.084897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:43.084897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:43.084897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:43.084897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:43.084897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:43.084897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:43.084897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:43.084897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:26:43.084897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:26:43.084897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:26:43.084897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:26:43.085978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:43.085978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:43.085978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:26:43.085978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:26:43.085978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:26:43.085978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:26:43.085978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:43.085978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:43.085978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:43.085978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:43.085978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:43.085978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:43.085978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:43.085978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:43.085978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:43.085978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:43.085978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:43.085978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:43.086958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:43.086958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:43.086958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:26:43.086958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:26:43.086958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:26:43.086958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:26:43.086958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:26:43.086958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:26:43.086958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 12:26:43.086958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:26:43.086958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 623/1239] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:26:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:43.092864 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc56ac-316d-4047-ac87-ce07779f3dcb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:43.196311 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc56ac-316d-4047-ac87-ce07779f3dcb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:43.229520 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91a5ff-d14c-4d4b-a049-c8e536d12159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:43.230118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91a5ff-d14c-4d4b-a049-c8e536d12159 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:43.230395 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91a5ff-d14c-4d4b-a049-c8e536d12159 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:43.230666 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91a5ff-d14c-4d4b-a049-c8e536d12159 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:43.232174 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91a5ff-d14c-4d4b-a049-c8e536d12159 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:43.232481 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91a5ff-d14c-4d4b-a049-c8e536d12159 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:43.234001 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91a5ff-d14c-4d4b-a049-c8e536d12159 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:43.250824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 617/1240] 10.222.0.49 () {56 vars in 1143 bytes} [Wed Aug 10 12:26:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:43.256882 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db41a1d-6972-4965-86a2-14b67c819057 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:43.357442 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db41a1d-6972-4965-86a2-14b67c819057 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:43.390889 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949a895-f637-46e2-bb4d-2d56dbd80c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:43.391526 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949a895-f637-46e2-bb4d-2d56dbd80c19 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:43.391816 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949a895-f637-46e2-bb4d-2d56dbd80c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:43.392091 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949a895-f637-46e2-bb4d-2d56dbd80c19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:43.452426 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:43.458893 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5949a895-f637-46e2-bb4d-2d56dbd80c19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:43.460376 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5949a895-f637-46e2-bb4d-2d56dbd80c19 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:43.555555 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 624/1241] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:26:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:26:43.561419 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91a5ff-d14c-4d4b-a049-c8e536d12159 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:43.589161 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:43.663262 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91a5ff-d14c-4d4b-a049-c8e536d12159 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:43.696282 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb8542f-cf45-49d8-96f2-c122412abca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:43.696921 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8542f-cf45-49d8-96f2-c122412abca8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:43.697210 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8542f-cf45-49d8-96f2-c122412abca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:43.697490 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8542f-cf45-49d8-96f2-c122412abca8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:43.708070 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb8542f-cf45-49d8-96f2-c122412abca8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:43.709505 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb8542f-cf45-49d8-96f2-c122412abca8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:43.725682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-4fb8542f-cf45-49d8-96f2-c122412abca8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:26:43.725682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:43.725682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:43.725682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:43.725682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:43.725682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:43.725682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:43.725682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:43.725682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:43.725682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:43.725682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:43.725682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:43.725682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:26:43.725682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:26:43.725682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:26:43.725682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:26:43.732875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:43.732875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:43.732875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:43.732875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:43.732875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:43.732875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:43.732875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:43.732875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:43.732875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:43.732875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:43.732875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:43.732875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:43.732875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:43.732875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:43.732875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:43.732875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:43.732875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:26:43.734194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:26:43.734194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:26:43.734194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:26:43.734194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 618/1242] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:26:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:43.734519 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949a895-f637-46e2-bb4d-2d56dbd80c19 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:43.829025 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:43.834959 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949a895-f637-46e2-bb4d-2d56dbd80c19 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:43.868201 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baff37f-9569-47fd-b5d4-bb36fad85abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:43.868859 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baff37f-9569-47fd-b5d4-bb36fad85abc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:43.869136 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baff37f-9569-47fd-b5d4-bb36fad85abc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:43.869404 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baff37f-9569-47fd-b5d4-bb36fad85abc None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:43.870030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baff37f-9569-47fd-b5d4-bb36fad85abc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:43.870331 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baff37f-9569-47fd-b5d4-bb36fad85abc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:43.872324 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baff37f-9569-47fd-b5d4-bb36fad85abc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:43.883389 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 625/1243] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:26:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:26:43.889968 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb8542f-cf45-49d8-96f2-c122412abca8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:43.968868 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:43.989836 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb8542f-cf45-49d8-96f2-c122412abca8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:44.023986 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf698e6-02ee-4a47-9aa7-688053a4d820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:44.024695 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf698e6-02ee-4a47-9aa7-688053a4d820 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:44.025005 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf698e6-02ee-4a47-9aa7-688053a4d820 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:44.025317 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf698e6-02ee-4a47-9aa7-688053a4d820 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:44.031939 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf698e6-02ee-4a47-9aa7-688053a4d820 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:44.033380 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf698e6-02ee-4a47-9aa7-688053a4d820 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:44.044420 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-bcf698e6-02ee-4a47-9aa7-688053a4d820 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:26:44.044420 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:44.044420 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:44.044420 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:44.044420 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:44.044420 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:44.044420 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:44.044420 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:44.044420 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:44.044420 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:44.044420 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:44.044420 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:44.044420 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:26:44.044420 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:26:44.044420 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:26:44.045556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:26:44.045556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:44.045556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:44.045556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:44.045556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:44.045556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:44.045556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:44.045556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:44.045556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:44.045556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:44.045556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:44.045556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:44.045556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:44.045556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:44.045556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:44.045556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:44.045556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:44.046645 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:26:44.046645 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:26:44.046645 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:26:44.046645 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:26:44.046645 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:26:44.046645 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:26:44.046645 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:26:44.046645 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:26:44.046645 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 619/1244] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:26:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:44.051542 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baff37f-9569-47fd-b5d4-bb36fad85abc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:44.151714 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baff37f-9569-47fd-b5d4-bb36fad85abc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:44.185690 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8db55-b5b4-40ca-a7b5-15a6425a63e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:44.186284 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8db55-b5b4-40ca-a7b5-15a6425a63e0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:44.186556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8db55-b5b4-40ca-a7b5-15a6425a63e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:44.186821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8db55-b5b4-40ca-a7b5-15a6425a63e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:44.187527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8db55-b5b4-40ca-a7b5-15a6425a63e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:44.187828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8db55-b5b4-40ca-a7b5-15a6425a63e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:44.189239 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8db55-b5b4-40ca-a7b5-15a6425a63e0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:44.200764 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 626/1245] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:26:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:44.206858 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf698e6-02ee-4a47-9aa7-688053a4d820 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:44.305404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf698e6-02ee-4a47-9aa7-688053a4d820 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:44.337941 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd9eea-77f6-4937-bce6-ec91226db8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:44.338534 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd9eea-77f6-4937-bce6-ec91226db8ee None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:44.338811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd9eea-77f6-4937-bce6-ec91226db8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:44.339079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd9eea-77f6-4937-bce6-ec91226db8ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:44.339820 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dd9eea-77f6-4937-bce6-ec91226db8ee None admin] RBAC: Adding query filter params (role.id=a9bbb011afca4176a3def9356cb5f165, scope.project.id=8e201e5f281a4977be503e22daf149d7, user.id=3ff152e22374407eb72caf648a763107) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:44.340127 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dd9eea-77f6-4937-bce6-ec91226db8ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:44.341835 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dd9eea-77f6-4937-bce6-ec91226db8ee None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:44.347497 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 620/1246] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:26:44 2022] GET /identity/v3/role_assignments?user.id=3ff152e22374407eb72caf648a763107&scope.project.id=8e201e5f281a4977be503e22daf149d7&role.id=a9bbb011afca4176a3def9356cb5f165 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:46.373778 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0d6c9-55af-404d-a740-7305b456d1c7 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:46.374297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0d6c9-55af-404d-a740-7305b456d1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:46.374297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0d6c9-55af-404d-a740-7305b456d1c7 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:46.375935 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 627/1247] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:26:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:26:46.380995 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594a366-5a95-4637-a9ee-c1eac456b286 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:46.383657 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594a366-5a95-4637-a9ee-c1eac456b286 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:46.383657 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594a366-5a95-4637-a9ee-c1eac456b286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:46.388068 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:46.400495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7594a366-5a95-4637-a9ee-c1eac456b286 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:26:46.401685 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7594a366-5a95-4637-a9ee-c1eac456b286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:46.467997 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 621/1248] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:26:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:26:46.491324 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0d6c9-55af-404d-a740-7305b456d1c7 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:46.510393 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:46.593828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0d6c9-55af-404d-a740-7305b456d1c7 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:46.628060 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac96040-affe-4a80-ae3f-e4923932f279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:46.628738 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac96040-affe-4a80-ae3f-e4923932f279 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:46.629014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac96040-affe-4a80-ae3f-e4923932f279 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:46.629287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac96040-affe-4a80-ae3f-e4923932f279 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:46.639343 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac96040-affe-4a80-ae3f-e4923932f279 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:26:46.644129 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-9ac96040-affe-4a80-ae3f-e4923932f279 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:26:46.644129 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:46.644129 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:46.644129 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:46.644129 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:46.644129 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:46.644129 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:46.644129 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:46.644129 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:46.644129 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:46.644129 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:46.644129 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:46.644129 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:26:46.644129 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:26:46.644129 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:26:46.644129 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:26:46.645214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:26:46.645214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:26:46.645214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:26:46.645214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:26:46.645214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:46.645214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:46.645214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:46.645214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:46.645214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:46.645214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:46.645214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:46.645214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:46.645214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:46.645214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:46.645214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:46.645214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:46.646246 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:46.646246 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:46.646246 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:46.646246 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:46.646246 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:26:46.646246 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:26:46.646246 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:26:46.646246 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:26:46.646246 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:26:46.646246 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:26:46.646246 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:26:46.646246 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:26:46.646246 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 628/1249] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:26:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:46.651621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7594a366-5a95-4637-a9ee-c1eac456b286 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:46.750786 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7594a366-5a95-4637-a9ee-c1eac456b286 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:46.780583 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:46.785346 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9fd34b-c81f-4769-8ea9-a02df44d5b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:46.785948 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fd34b-c81f-4769-8ea9-a02df44d5b4c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:46.786231 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fd34b-c81f-4769-8ea9-a02df44d5b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:46.786506 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fd34b-c81f-4769-8ea9-a02df44d5b4c None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:46.787157 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9fd34b-c81f-4769-8ea9-a02df44d5b4c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:46.787493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9fd34b-c81f-4769-8ea9-a02df44d5b4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:46.788848 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9fd34b-c81f-4769-8ea9-a02df44d5b4c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:46.796711 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 622/1250] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:26:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:46.802469 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac96040-affe-4a80-ae3f-e4923932f279 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:46.880493 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:46.902906 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac96040-affe-4a80-ae3f-e4923932f279 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:46.936760 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7499be-01db-4ea7-9eed-8c28706ec456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:46.937641 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7499be-01db-4ea7-9eed-8c28706ec456 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:46.938099 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7499be-01db-4ea7-9eed-8c28706ec456 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:46.938745 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7499be-01db-4ea7-9eed-8c28706ec456 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:46.948298 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7499be-01db-4ea7-9eed-8c28706ec456 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:46.949718 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7499be-01db-4ea7-9eed-8c28706ec456 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:46.964956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-9b7499be-01db-4ea7-9eed-8c28706ec456 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:26:46.964956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:46.964956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:46.964956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:46.964956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:46.964956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:46.964956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:46.964956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:46.964956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:46.964956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:46.964956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:46.964956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:46.964956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:26:46.964956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:26:46.964956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:26:46.964956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:26:46.966170 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:46.966170 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:46.966170 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:46.966170 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:46.966170 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:46.966170 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:46.966170 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:46.966170 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:46.966170 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:46.966170 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:46.966170 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:46.966170 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:46.966170 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:46.966170 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:46.966170 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:46.966170 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:46.966170 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:26:46.967468 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:26:46.967468 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:26:46.967468 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:26:46.967468 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 629/1251] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:26:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:46.972597 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9fd34b-c81f-4769-8ea9-a02df44d5b4c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:47.070631 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9fd34b-c81f-4769-8ea9-a02df44d5b4c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:47.104007 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73037071-8648-4ec2-aad0-81a187512278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:47.104613 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73037071-8648-4ec2-aad0-81a187512278 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:47.104893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73037071-8648-4ec2-aad0-81a187512278 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:47.105168 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73037071-8648-4ec2-aad0-81a187512278 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:47.105802 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73037071-8648-4ec2-aad0-81a187512278 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:47.106619 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73037071-8648-4ec2-aad0-81a187512278 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:47.108092 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73037071-8648-4ec2-aad0-81a187512278 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:47.119613 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 623/1252] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:26:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:26:47.126297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7499be-01db-4ea7-9eed-8c28706ec456 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:47.226275 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7499be-01db-4ea7-9eed-8c28706ec456 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:47.252652 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:47.260383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06c790-8689-45c4-8785-8369bdf8fe96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:47.261005 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06c790-8689-45c4-8785-8369bdf8fe96 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:47.261279 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06c790-8689-45c4-8785-8369bdf8fe96 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:47.261551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06c790-8689-45c4-8785-8369bdf8fe96 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:47.270104 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06c790-8689-45c4-8785-8369bdf8fe96 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:47.271559 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06c790-8689-45c4-8785-8369bdf8fe96 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:47.284145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-3d06c790-8689-45c4-8785-8369bdf8fe96 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 12:26:47.284145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:47.284145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:47.284145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:47.284145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:47.284145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:47.284145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:47.284145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:47.284145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:47.284145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:47.284145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:47.284145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:47.284145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:26:47.284145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:26:47.284145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:26:47.284145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:26:47.290494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:47.290494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:47.290494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:26:47.290494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:26:47.290494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:26:47.290494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:26:47.290494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:47.290494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:47.290494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:47.290494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:47.290494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:47.290494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:47.290494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:47.290494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:47.290494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:47.290494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:47.290494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:47.290494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:47.291741 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:47.291741 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:47.291741 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:26:47.291741 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:26:47.291741 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:26:47.291741 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:26:47.291741 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:26:47.291741 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:26:47.291741 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 12:26:47.291741 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:26:47.291741 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 630/1253] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:26:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:47.292595 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73037071-8648-4ec2-aad0-81a187512278 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:47.346131 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:47.389787 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73037071-8648-4ec2-aad0-81a187512278 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:47.423957 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28e832-a719-445d-a504-f2a0b4804674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:47.424568 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28e832-a719-445d-a504-f2a0b4804674 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:47.424853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28e832-a719-445d-a504-f2a0b4804674 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:47.425130 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28e832-a719-445d-a504-f2a0b4804674 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:47.425851 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28e832-a719-445d-a504-f2a0b4804674 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:47.426162 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28e832-a719-445d-a504-f2a0b4804674 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:47.427640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28e832-a719-445d-a504-f2a0b4804674 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:47.444428 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 624/1254] 10.222.0.49 () {56 vars in 1143 bytes} [Wed Aug 10 12:26:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:47.450897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06c790-8689-45c4-8785-8369bdf8fe96 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:47.550837 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06c790-8689-45c4-8785-8369bdf8fe96 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:47.585222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d2ca26-5948-484e-a5c0-c4a38f2a57c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:47.585814 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2ca26-5948-484e-a5c0-c4a38f2a57c1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:47.586088 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2ca26-5948-484e-a5c0-c4a38f2a57c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:47.586357 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2ca26-5948-484e-a5c0-c4a38f2a57c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:47.640653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:47.652280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d2ca26-5948-484e-a5c0-c4a38f2a57c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:47.654351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d2ca26-5948-484e-a5c0-c4a38f2a57c1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:47.751611 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 631/1255] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:26:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:26:47.757621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28e832-a719-445d-a504-f2a0b4804674 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:47.858214 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28e832-a719-445d-a504-f2a0b4804674 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:47.870945 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:47.892358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d78ff-2150-412d-87a7-94f81e80e9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:47.892981 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d78ff-2150-412d-87a7-94f81e80e9aa None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:47.893259 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d78ff-2150-412d-87a7-94f81e80e9aa None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:47.893528 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d78ff-2150-412d-87a7-94f81e80e9aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:47.903912 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d78ff-2150-412d-87a7-94f81e80e9aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:47.905343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d78ff-2150-412d-87a7-94f81e80e9aa None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:47.920609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d85d78ff-2150-412d-87a7-94f81e80e9aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:26:47.920609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:47.920609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:47.920609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:47.920609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:47.920609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:47.920609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:47.920609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:47.920609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:47.920609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:47.920609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:47.920609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:47.920609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:26:47.920609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:26:47.920609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:26:47.920609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:26:47.921626 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:47.921626 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:47.921626 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:47.921626 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:47.921626 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:47.921626 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:47.921626 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:47.921626 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:47.921626 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:47.921626 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:47.921626 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:47.921626 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:47.921626 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:47.921626 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:47.921626 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:47.921626 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:47.921626 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:26:47.929146 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:26:47.929146 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:26:47.929146 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:26:47.929146 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 625/1256] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:26:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:47.929475 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d2ca26-5948-484e-a5c0-c4a38f2a57c1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:48.014077 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:48.025902 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d2ca26-5948-484e-a5c0-c4a38f2a57c1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:48.059524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07f94a-02fb-47e8-9a22-9173c545c4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:48.060119 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07f94a-02fb-47e8-9a22-9173c545c4eb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:48.060425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07f94a-02fb-47e8-9a22-9173c545c4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:48.060723 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07f94a-02fb-47e8-9a22-9173c545c4eb None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:48.061352 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07f94a-02fb-47e8-9a22-9173c545c4eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:48.061654 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07f94a-02fb-47e8-9a22-9173c545c4eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:48.062997 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07f94a-02fb-47e8-9a22-9173c545c4eb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:48.074370 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 632/1257] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:26:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:48.080806 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d78ff-2150-412d-87a7-94f81e80e9aa None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:48.180509 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d78ff-2150-412d-87a7-94f81e80e9aa None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:48.213193 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa2117-3943-43c6-b782-164635215dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:48.213817 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa2117-3943-43c6-b782-164635215dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:48.214113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa2117-3943-43c6-b782-164635215dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:48.214377 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa2117-3943-43c6-b782-164635215dd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:48.221472 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa2117-3943-43c6-b782-164635215dd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:48.222918 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa2117-3943-43c6-b782-164635215dd9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:48.234736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-5efa2117-3943-43c6-b782-164635215dd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:26:48.234736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:48.234736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:48.234736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:48.234736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:48.234736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:48.234736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:48.234736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:48.234736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:48.234736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:48.234736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:48.234736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:48.234736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:26:48.234736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:26:48.234736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:26:48.235846 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:26:48.235846 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:48.235846 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:48.235846 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:48.235846 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:48.235846 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:48.235846 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:48.235846 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:48.235846 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:48.235846 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:48.235846 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:48.235846 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:48.235846 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:48.235846 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:48.235846 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:48.235846 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:48.235846 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:48.236827 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:26:48.236827 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:26:48.236827 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:26:48.236827 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:26:48.236827 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:26:48.236827 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:26:48.236827 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:26:48.236827 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:26:48.236827 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 626/1258] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:26:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:48.241802 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07f94a-02fb-47e8-9a22-9173c545c4eb None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:48.341887 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07f94a-02fb-47e8-9a22-9173c545c4eb None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:48.374623 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9638bf52-0553-48bc-8856-e42d7f34b9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:48.375518 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638bf52-0553-48bc-8856-e42d7f34b9bb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:48.375802 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638bf52-0553-48bc-8856-e42d7f34b9bb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:48.376076 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638bf52-0553-48bc-8856-e42d7f34b9bb None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:48.376759 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9638bf52-0553-48bc-8856-e42d7f34b9bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:48.377066 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9638bf52-0553-48bc-8856-e42d7f34b9bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:48.378485 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9638bf52-0553-48bc-8856-e42d7f34b9bb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:48.390037 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 633/1259] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:26:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:48.396203 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa2117-3943-43c6-b782-164635215dd9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:48.422300 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:48.494624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa2117-3943-43c6-b782-164635215dd9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:48.529316 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3c4904-04f0-4233-86ee-2e803f5110f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:48.529923 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c4904-04f0-4233-86ee-2e803f5110f7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:48.530207 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c4904-04f0-4233-86ee-2e803f5110f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:48.530479 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c4904-04f0-4233-86ee-2e803f5110f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:48.531223 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3c4904-04f0-4233-86ee-2e803f5110f7 None admin] RBAC: Adding query filter params (role.id=bffbe227cda1418cbb58394ddff6f861, scope.project.id=8e201e5f281a4977be503e22daf149d7, user.id=3ff152e22374407eb72caf648a763107) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:48.531553 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3c4904-04f0-4233-86ee-2e803f5110f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:48.532955 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3c4904-04f0-4233-86ee-2e803f5110f7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:48.538265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 627/1260] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:26:48 2022] GET /identity/v3/role_assignments?user.id=3ff152e22374407eb72caf648a763107&scope.project.id=8e201e5f281a4977be503e22daf149d7&role.id=bffbe227cda1418cbb58394ddff6f861 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:26:50.537314 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97750100-bcc6-4fb4-ad39-b21d36db4052 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:50.537838 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97750100-bcc6-4fb4-ad39-b21d36db4052 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:50.537838 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97750100-bcc6-4fb4-ad39-b21d36db4052 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:50.539125 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 634/1261] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:26:50.544880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9daec7-ce76-4534-bf08-b73bd5328841 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:50.545301 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9daec7-ce76-4534-bf08-b73bd5328841 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:50.545425 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9daec7-ce76-4534-bf08-b73bd5328841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:50.562990 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6a9daec7-ce76-4534-bf08-b73bd5328841 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:26:50.564233 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6a9daec7-ce76-4534-bf08-b73bd5328841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:50.631703 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 628/1262] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:26:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:26:50.655357 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97750100-bcc6-4fb4-ad39-b21d36db4052 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:50.666302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:50.753542 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97750100-bcc6-4fb4-ad39-b21d36db4052 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:50.788116 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc2cc6-9588-4580-bff7-ac61eba22c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:50.788680 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc2cc6-9588-4580-bff7-ac61eba22c1e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:50.788956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc2cc6-9588-4580-bff7-ac61eba22c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:50.789225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc2cc6-9588-4580-bff7-ac61eba22c1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:50.800059 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dc2cc6-9588-4580-bff7-ac61eba22c1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:26:50.804636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-b8dc2cc6-9588-4580-bff7-ac61eba22c1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:26:50.804636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:50.804636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:50.804636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:50.804636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:50.804636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:50.804636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:50.804636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:50.804636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:50.804636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:50.804636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:50.804636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:50.804636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:26:50.804636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:26:50.804636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:26:50.804636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:26:50.809829 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:26:50.809829 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:26:50.809829 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:26:50.809829 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:26:50.809829 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:50.809829 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:50.809829 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:50.809829 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:50.809829 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:50.809829 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:50.809829 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:50.809829 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:50.809829 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:50.809829 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:50.809829 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:50.809829 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:50.811005 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:50.811005 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:50.811005 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:50.811005 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:50.811005 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:26:50.811005 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:26:50.811005 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:26:50.811005 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:26:50.811005 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:26:50.811005 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:26:50.811005 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:26:50.811005 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:26:50.811005 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 635/1263] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:26:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:50.811965 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9daec7-ce76-4534-bf08-b73bd5328841 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:50.829954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:50.908409 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9daec7-ce76-4534-bf08-b73bd5328841 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:50.943506 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b572abe-6469-4a7d-84c8-bcd5e165dfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:50.944102 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b572abe-6469-4a7d-84c8-bcd5e165dfec None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:50.944381 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b572abe-6469-4a7d-84c8-bcd5e165dfec None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:50.944652 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b572abe-6469-4a7d-84c8-bcd5e165dfec None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:50.945300 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b572abe-6469-4a7d-84c8-bcd5e165dfec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:50.945599 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b572abe-6469-4a7d-84c8-bcd5e165dfec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:50.947102 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b572abe-6469-4a7d-84c8-bcd5e165dfec None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:50.955716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 629/1264] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:26:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:50.961730 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc2cc6-9588-4580-bff7-ac61eba22c1e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:51.026638 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:51.059317 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc2cc6-9588-4580-bff7-ac61eba22c1e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:51.091608 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017e748-8b41-4fd7-b582-ad72d96ead41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:51.092225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017e748-8b41-4fd7-b582-ad72d96ead41 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:51.092501 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017e748-8b41-4fd7-b582-ad72d96ead41 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:51.092771 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017e748-8b41-4fd7-b582-ad72d96ead41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:51.102923 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017e748-8b41-4fd7-b582-ad72d96ead41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:51.104382 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017e748-8b41-4fd7-b582-ad72d96ead41 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:51.118673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-a017e748-8b41-4fd7-b582-ad72d96ead41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:26:51.118673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:51.118673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:51.118673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:51.118673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:51.118673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:51.118673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:51.118673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:51.118673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:51.118673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:51.118673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:51.118673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:51.118673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:26:51.118673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:26:51.118673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:26:51.118673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:26:51.122946 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:51.122946 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:51.122946 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:51.122946 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:51.122946 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:51.122946 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:51.122946 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:51.122946 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:51.122946 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:51.122946 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:51.122946 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:51.122946 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:51.122946 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:51.122946 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:51.122946 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:51.122946 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:51.122946 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:26:51.124099 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:26:51.124099 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:26:51.124099 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:26:51.124099 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 636/1265] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:26:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:51.125908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b572abe-6469-4a7d-84c8-bcd5e165dfec None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:51.207054 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:51.226322 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b572abe-6469-4a7d-84c8-bcd5e165dfec None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:51.260427 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163d863-e87a-4d33-97a7-b9205230de4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:51.261039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163d863-e87a-4d33-97a7-b9205230de4b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:51.261314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163d863-e87a-4d33-97a7-b9205230de4b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:51.261578 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163d863-e87a-4d33-97a7-b9205230de4b None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:51.262204 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163d863-e87a-4d33-97a7-b9205230de4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:51.263154 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163d863-e87a-4d33-97a7-b9205230de4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:51.264556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163d863-e87a-4d33-97a7-b9205230de4b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:51.275995 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 630/1266] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:51.281771 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017e748-8b41-4fd7-b582-ad72d96ead41 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:51.378369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017e748-8b41-4fd7-b582-ad72d96ead41 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:51.397527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:51.410796 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a7e20-7fd4-4ab5-bf92-645f5c03ec52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:51.411446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a7e20-7fd4-4ab5-bf92-645f5c03ec52 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:51.411724 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a7e20-7fd4-4ab5-bf92-645f5c03ec52 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:51.411990 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a7e20-7fd4-4ab5-bf92-645f5c03ec52 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:51.419226 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a7e20-7fd4-4ab5-bf92-645f5c03ec52 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:51.420651 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a7e20-7fd4-4ab5-bf92-645f5c03ec52 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:51.431322 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-703a7e20-7fd4-4ab5-bf92-645f5c03ec52 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 12:26:51.431322 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:51.431322 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:51.431322 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:51.431322 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:51.431322 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:51.431322 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:51.431322 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:51.431322 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:51.431322 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:51.431322 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:51.431322 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:51.431322 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:26:51.431322 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:26:51.431322 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:26:51.431322 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:26:51.432446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:51.432446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:51.432446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:26:51.432446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:26:51.432446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:26:51.432446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:26:51.432446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:51.432446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:51.432446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:51.432446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:51.432446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:51.432446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:51.432446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:51.432446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:51.432446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:51.432446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:51.432446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:51.432446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:51.433804 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:51.433804 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:51.433804 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:26:51.433804 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:26:51.433804 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:26:51.433804 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:26:51.433804 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:26:51.433804 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:26:51.433804 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 12:26:51.433804 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:26:51.433804 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 637/1267] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:26:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:51.438973 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163d863-e87a-4d33-97a7-b9205230de4b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:51.537699 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163d863-e87a-4d33-97a7-b9205230de4b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:51.570930 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73c02a-682e-4534-8838-1d6d4d3b80a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:51.571623 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73c02a-682e-4534-8838-1d6d4d3b80a9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:51.571836 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73c02a-682e-4534-8838-1d6d4d3b80a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:51.572110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73c02a-682e-4534-8838-1d6d4d3b80a9 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:51.572837 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c73c02a-682e-4534-8838-1d6d4d3b80a9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:51.573146 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c73c02a-682e-4534-8838-1d6d4d3b80a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:51.574563 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c73c02a-682e-4534-8838-1d6d4d3b80a9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:51.584114 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:51.592187 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 631/1268] 10.222.0.49 () {56 vars in 1143 bytes} [Wed Aug 10 12:26:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:51.597939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a7e20-7fd4-4ab5-bf92-645f5c03ec52 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:51.699261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a7e20-7fd4-4ab5-bf92-645f5c03ec52 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:51.732404 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df548df8-aa7a-4a19-8fc3-61a35bbce19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:51.733032 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df548df8-aa7a-4a19-8fc3-61a35bbce19e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:51.733308 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df548df8-aa7a-4a19-8fc3-61a35bbce19e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:51.733579 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df548df8-aa7a-4a19-8fc3-61a35bbce19e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:51.743971 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df548df8-aa7a-4a19-8fc3-61a35bbce19e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:51.745394 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df548df8-aa7a-4a19-8fc3-61a35bbce19e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:51.759847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-df548df8-aa7a-4a19-8fc3-61a35bbce19e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:26:51.759847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:51.759847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:51.759847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:51.759847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:51.759847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:51.759847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:51.759847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:51.759847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:51.759847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:51.759847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:51.759847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:51.759847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:26:51.759847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:26:51.759847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:26:51.759847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:26:51.760999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:51.760999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:51.760999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:51.760999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:51.760999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:51.760999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:51.760999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:51.760999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:51.760999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:51.760999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:51.760999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:51.760999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:51.760999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:51.760999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:51.760999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:51.760999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:51.760999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:26:51.762267 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:26:51.762267 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:26:51.762267 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:26:51.762267 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 638/1269] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:26:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:51.766734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73c02a-682e-4534-8838-1d6d4d3b80a9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:51.865063 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73c02a-682e-4534-8838-1d6d4d3b80a9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:51.898729 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c82998-8908-462f-8460-7e4136fa4979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:51.899362 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c82998-8908-462f-8460-7e4136fa4979 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:51.899640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c82998-8908-462f-8460-7e4136fa4979 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:51.899911 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c82998-8908-462f-8460-7e4136fa4979 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:51.900540 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c82998-8908-462f-8460-7e4136fa4979 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:51.900839 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c82998-8908-462f-8460-7e4136fa4979 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:51.902299 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c82998-8908-462f-8460-7e4136fa4979 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:51.913319 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 632/1270] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:51.919657 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df548df8-aa7a-4a19-8fc3-61a35bbce19e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:51.950925 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:52.018634 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df548df8-aa7a-4a19-8fc3-61a35bbce19e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:52.053693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ffe65-fa06-4e0b-9673-75fa82e5b7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:52.054318 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ffe65-fa06-4e0b-9673-75fa82e5b7ee None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:52.054597 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ffe65-fa06-4e0b-9673-75fa82e5b7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:52.054897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ffe65-fa06-4e0b-9673-75fa82e5b7ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:52.061372 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ffe65-fa06-4e0b-9673-75fa82e5b7ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:52.062796 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ffe65-fa06-4e0b-9673-75fa82e5b7ee None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:52.073642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-d37ffe65-fa06-4e0b-9673-75fa82e5b7ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:26:52.073642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:52.073642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:52.073642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:52.073642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:52.073642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:52.073642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:52.073642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:52.073642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:52.073642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:52.073642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:52.073642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:52.073642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:26:52.073642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:26:52.073642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:26:52.074704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:26:52.074704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:52.074704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:52.074704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:52.074704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:52.074704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:52.074704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:52.074704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:52.074704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:52.074704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:52.074704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:52.074704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:52.074704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:52.074704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:52.074704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:52.074704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:52.074704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:52.075680 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:26:52.075680 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:26:52.075680 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:26:52.075680 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:26:52.075680 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:26:52.075680 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:26:52.075680 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:26:52.075680 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:26:52.075680 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 639/1271] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:26:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:52.080619 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c82998-8908-462f-8460-7e4136fa4979 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:52.141452 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:52.181037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c82998-8908-462f-8460-7e4136fa4979 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:52.213580 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c7f08-839e-44c6-bedd-7d44821bf446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:52.214118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c7f08-839e-44c6-bedd-7d44821bf446 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:52.214399 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c7f08-839e-44c6-bedd-7d44821bf446 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:52.214671 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c7f08-839e-44c6-bedd-7d44821bf446 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:52.215388 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415c7f08-839e-44c6-bedd-7d44821bf446 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:52.220901 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415c7f08-839e-44c6-bedd-7d44821bf446 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:52.220901 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415c7f08-839e-44c6-bedd-7d44821bf446 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:52.229788 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 633/1272] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:26:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:52.236074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ffe65-fa06-4e0b-9673-75fa82e5b7ee None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:52.330469 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:52.338803 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ffe65-fa06-4e0b-9673-75fa82e5b7ee None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:52.372466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337fcc1-ac52-404c-840e-14f2b1f8e2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:52.373240 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337fcc1-ac52-404c-840e-14f2b1f8e2f3 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:52.373521 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337fcc1-ac52-404c-840e-14f2b1f8e2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:52.373798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337fcc1-ac52-404c-840e-14f2b1f8e2f3 None admin] PATH_INFO: `/v3/projects/8e201e5f281a4977be503e22daf149d7/users/3ff152e22374407eb72caf648a763107/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:52.375987 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337fcc1-ac52-404c-840e-14f2b1f8e2f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e201e5f281a4977be503e22daf149d7, user_id=3ff152e22374407eb72caf648a763107, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:52.377411 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337fcc1-ac52-404c-840e-14f2b1f8e2f3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:52.450009 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 640/1273] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 12:26:52 2022] PUT /identity/v3/projects/8e201e5f281a4977be503e22daf149d7/users/3ff152e22374407eb72caf648a763107/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:26:54.466732 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3439fe4-d5ba-4286-a6a3-dc8ec3f2d7f8 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:54.467451 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3439fe4-d5ba-4286-a6a3-dc8ec3f2d7f8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:54.467451 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3439fe4-d5ba-4286-a6a3-dc8ec3f2d7f8 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:54.468693 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 634/1274] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:26:54.473804 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d339b2d-c556-4e3d-af4a-277af800287f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:54.474253 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d339b2d-c556-4e3d-af4a-277af800287f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:54.474356 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d339b2d-c556-4e3d-af4a-277af800287f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:54.492658 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3d339b2d-c556-4e3d-af4a-277af800287f None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:26:54.493843 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3d339b2d-c556-4e3d-af4a-277af800287f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:54.560592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 641/1275] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:26:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:26:54.584322 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3439fe4-d5ba-4286-a6a3-dc8ec3f2d7f8 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:54.688736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3439fe4-d5ba-4286-a6a3-dc8ec3f2d7f8 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:54.705912 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:54.722308 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e1c220-ca79-4bbf-a0f5-05a907eeda85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:54.722927 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1c220-ca79-4bbf-a0f5-05a907eeda85 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:54.723236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1c220-ca79-4bbf-a0f5-05a907eeda85 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:54.723505 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1c220-ca79-4bbf-a0f5-05a907eeda85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:54.734115 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e1c220-ca79-4bbf-a0f5-05a907eeda85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:26:54.740565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-06e1c220-ca79-4bbf-a0f5-05a907eeda85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:26:54.740565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:54.740565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:54.740565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:54.740565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:54.740565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:54.740565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:54.740565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:54.740565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:54.740565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:54.740565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:54.740565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:54.740565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:26:54.740565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:26:54.740565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:26:54.740565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:26:54.741669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:26:54.741669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:26:54.741669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:26:54.741669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:26:54.741669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:54.741669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:54.741669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:54.741669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:54.741669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:54.741669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:54.741669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:54.741669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:54.741669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:54.741669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:54.741669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:54.741669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:54.742735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:54.742735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:54.742735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:54.742735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:54.742735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:26:54.742735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:26:54.742735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:26:54.742735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:26:54.742735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:26:54.742735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:26:54.742735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:26:54.742735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:26:54.742735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 635/1276] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 12:26:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:54.746437 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d339b2d-c556-4e3d-af4a-277af800287f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:54.830133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:54.847224 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d339b2d-c556-4e3d-af4a-277af800287f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:54.881208 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c14b94-ff5f-4a77-a0d2-7a4f249d07d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:54.881813 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c14b94-ff5f-4a77-a0d2-7a4f249d07d1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:54.882096 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c14b94-ff5f-4a77-a0d2-7a4f249d07d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:54.882377 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c14b94-ff5f-4a77-a0d2-7a4f249d07d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:54.883036 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c14b94-ff5f-4a77-a0d2-7a4f249d07d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:54.883374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c14b94-ff5f-4a77-a0d2-7a4f249d07d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:54.884730 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c14b94-ff5f-4a77-a0d2-7a4f249d07d1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:54.892854 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 642/1277] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 12:26:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:54.898760 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e1c220-ca79-4bbf-a0f5-05a907eeda85 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:54.996258 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e1c220-ca79-4bbf-a0f5-05a907eeda85 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:55.029821 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee7b59-4611-460a-ac3d-58922275ab01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:55.030428 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee7b59-4611-460a-ac3d-58922275ab01 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:55.030708 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee7b59-4611-460a-ac3d-58922275ab01 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:55.030977 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee7b59-4611-460a-ac3d-58922275ab01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:55.040642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee7b59-4611-460a-ac3d-58922275ab01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:55.042062 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee7b59-4611-460a-ac3d-58922275ab01 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:55.056728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-acee7b59-4611-460a-ac3d-58922275ab01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:26:55.056728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:55.056728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:55.056728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:55.056728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:55.056728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:55.056728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:55.056728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:55.056728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:55.056728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:55.056728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:55.056728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:55.056728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:26:55.056728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:26:55.056728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:26:55.056728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:26:55.058086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:55.058086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:55.058086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:55.058086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:55.058086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:55.058086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:55.058086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:55.058086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:55.058086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:55.058086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:55.058086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:55.058086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:55.058086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:55.058086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:55.058086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:55.058086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:55.058086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:26:55.059209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:26:55.059209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:26:55.059209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:26:55.059209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 636/1278] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:26:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:55.064437 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c14b94-ff5f-4a77-a0d2-7a4f249d07d1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:55.165329 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c14b94-ff5f-4a77-a0d2-7a4f249d07d1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:55.199394 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e7d3d0-debc-4984-996c-8d4fd99dd95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:55.200633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7d3d0-debc-4984-996c-8d4fd99dd95f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:55.200911 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7d3d0-debc-4984-996c-8d4fd99dd95f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:55.201334 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7d3d0-debc-4984-996c-8d4fd99dd95f None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:55.201966 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e7d3d0-debc-4984-996c-8d4fd99dd95f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:55.202271 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e7d3d0-debc-4984-996c-8d4fd99dd95f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:55.203654 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e7d3d0-debc-4984-996c-8d4fd99dd95f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:55.215192 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 643/1279] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:55.220941 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee7b59-4611-460a-ac3d-58922275ab01 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:55.255391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:55.325775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee7b59-4611-460a-ac3d-58922275ab01 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:55.360702 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a6a4f-5357-48cb-878d-9196cbfa42b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:55.361327 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a6a4f-5357-48cb-878d-9196cbfa42b2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:55.361610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a6a4f-5357-48cb-878d-9196cbfa42b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:55.361883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a6a4f-5357-48cb-878d-9196cbfa42b2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:55.369149 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a6a4f-5357-48cb-878d-9196cbfa42b2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:55.370593 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a6a4f-5357-48cb-878d-9196cbfa42b2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:55.383039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-4d1a6a4f-5357-48cb-878d-9196cbfa42b2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 12:26:55.383039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:55.383039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:55.383039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:55.383039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:55.383039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:55.383039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:55.383039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:55.383039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:55.383039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:55.383039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:55.383039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:55.383039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:26:55.383039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:26:55.383039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:26:55.383039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:26:55.388681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:55.388681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:55.388681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:26:55.388681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:26:55.388681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:26:55.388681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:26:55.388681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:55.388681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:55.388681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:55.388681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:55.388681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:55.388681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:55.388681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:55.388681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:55.388681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:55.388681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:55.388681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:55.388681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:55.389904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:55.389904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:55.389904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:26:55.389904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:26:55.389904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:26:55.389904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:26:55.389904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:26:55.389904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:26:55.389904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 12:26:55.389904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:26:55.389904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 637/1280] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:26:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:55.390652 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e7d3d0-debc-4984-996c-8d4fd99dd95f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:55.394056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:55.488330 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e7d3d0-debc-4984-996c-8d4fd99dd95f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:55.521892 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b5ad3-3330-4c8c-a08a-73c34aba880c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:55.522496 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b5ad3-3330-4c8c-a08a-73c34aba880c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:55.522773 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b5ad3-3330-4c8c-a08a-73c34aba880c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:55.523057 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b5ad3-3330-4c8c-a08a-73c34aba880c None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:55.523790 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b5ad3-3330-4c8c-a08a-73c34aba880c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:55.524098 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b5ad3-3330-4c8c-a08a-73c34aba880c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:55.525557 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b5ad3-3330-4c8c-a08a-73c34aba880c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:55.543338 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 644/1281] 10.222.0.49 () {56 vars in 1143 bytes} [Wed Aug 10 12:26:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:55.549768 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a6a4f-5357-48cb-878d-9196cbfa42b2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:55.639570 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:55.648800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a6a4f-5357-48cb-878d-9196cbfa42b2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:55.682965 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06612aaa-0cdb-4b7e-bd5b-1e88ddf0b976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:55.683599 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06612aaa-0cdb-4b7e-bd5b-1e88ddf0b976 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:55.683889 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06612aaa-0cdb-4b7e-bd5b-1e88ddf0b976 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:55.684220 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06612aaa-0cdb-4b7e-bd5b-1e88ddf0b976 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:55.752041 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06612aaa-0cdb-4b7e-bd5b-1e88ddf0b976 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:55.753505 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06612aaa-0cdb-4b7e-bd5b-1e88ddf0b976 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:55.841231 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:55.852119 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 638/1282] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 12:26:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:26:55.858797 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b5ad3-3330-4c8c-a08a-73c34aba880c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:55.921042 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:55.957332 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b5ad3-3330-4c8c-a08a-73c34aba880c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:55.991339 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b04362-d326-45f9-89d1-0117c753fe7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:55.991960 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b04362-d326-45f9-89d1-0117c753fe7b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:55.992238 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b04362-d326-45f9-89d1-0117c753fe7b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:55.992508 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b04362-d326-45f9-89d1-0117c753fe7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:56.004402 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b04362-d326-45f9-89d1-0117c753fe7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:56.004402 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b04362-d326-45f9-89d1-0117c753fe7b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:56.018699 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-91b04362-d326-45f9-89d1-0117c753fe7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:26:56.018699 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:56.018699 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:56.018699 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:56.018699 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:56.018699 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:56.018699 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:56.018699 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:56.018699 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:56.018699 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:56.018699 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:56.018699 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:56.018699 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:26:56.018699 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:26:56.018699 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:26:56.018699 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:26:56.019762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:56.019762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:56.019762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:56.019762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:56.019762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:56.019762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:56.019762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:56.019762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:56.019762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:56.019762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:56.019762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:56.019762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:56.019762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:56.019762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:56.019762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:56.019762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:56.019762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:26:56.020650 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:26:56.020650 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:26:56.020650 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:26:56.020650 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 645/1283] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:26:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:56.025530 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06612aaa-0cdb-4b7e-bd5b-1e88ddf0b976 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:56.123307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06612aaa-0cdb-4b7e-bd5b-1e88ddf0b976 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:56.157588 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f09fd-cd49-49da-9f7e-21b5b93bce7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:56.158192 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f09fd-cd49-49da-9f7e-21b5b93bce7d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:56.158472 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f09fd-cd49-49da-9f7e-21b5b93bce7d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:56.158744 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f09fd-cd49-49da-9f7e-21b5b93bce7d None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:56.159412 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f09fd-cd49-49da-9f7e-21b5b93bce7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:56.159716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f09fd-cd49-49da-9f7e-21b5b93bce7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:56.161102 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f09fd-cd49-49da-9f7e-21b5b93bce7d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:56.172268 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 639/1284] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 12:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:56.177909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b04362-d326-45f9-89d1-0117c753fe7b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:56.275775 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b04362-d326-45f9-89d1-0117c753fe7b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:56.289876 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:56.310203 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1648ac8e-fddd-4dcc-8ddc-069dca2ad4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:56.310963 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648ac8e-fddd-4dcc-8ddc-069dca2ad4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:56.311270 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648ac8e-fddd-4dcc-8ddc-069dca2ad4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:56.311543 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648ac8e-fddd-4dcc-8ddc-069dca2ad4b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:56.317829 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1648ac8e-fddd-4dcc-8ddc-069dca2ad4b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:56.319351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1648ac8e-fddd-4dcc-8ddc-069dca2ad4b8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:56.330297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-1648ac8e-fddd-4dcc-8ddc-069dca2ad4b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:26:56.330297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:56.330297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:56.330297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:56.330297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:56.330297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:56.330297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:56.330297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:56.330297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:56.330297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:56.330297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:56.330297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:56.330297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:26:56.330297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:26:56.330297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:26:56.331386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:26:56.331386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:56.331386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:56.331386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:56.331386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:56.331386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:56.331386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:56.331386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:56.331386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:56.331386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:56.331386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:56.331386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:56.331386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:56.331386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:56.331386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:56.331386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:56.331386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:56.332528 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:26:56.332528 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:26:56.332528 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:26:56.332528 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:26:56.332528 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:26:56.332528 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:26:56.332528 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:26:56.332528 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Aug 10 12:26:56.332528 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 646/1285] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 12:26:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:26:56.337564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f09fd-cd49-49da-9f7e-21b5b93bce7d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:56.387283 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:56.438353 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f09fd-cd49-49da-9f7e-21b5b93bce7d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:56.471709 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d4abe-f472-4c64-be7a-5e8469d9c714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:56.472311 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d4abe-f472-4c64-be7a-5e8469d9c714 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:56.472592 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d4abe-f472-4c64-be7a-5e8469d9c714 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:56.472866 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d4abe-f472-4c64-be7a-5e8469d9c714 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:56.473547 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d4abe-f472-4c64-be7a-5e8469d9c714 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:56.473906 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d4abe-f472-4c64-be7a-5e8469d9c714 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:56.475414 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d4abe-f472-4c64-be7a-5e8469d9c714 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:56.487793 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 640/1286] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 12:26:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:56.494040 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1648ac8e-fddd-4dcc-8ddc-069dca2ad4b8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:56.594600 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1648ac8e-fddd-4dcc-8ddc-069dca2ad4b8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:56.628041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c3f41-d2b4-4133-9e63-45d887eda38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:56.628642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c3f41-d2b4-4133-9e63-45d887eda38b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:56.628926 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c3f41-d2b4-4133-9e63-45d887eda38b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:56.629196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c3f41-d2b4-4133-9e63-45d887eda38b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:56.629908 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7c3f41-d2b4-4133-9e63-45d887eda38b None admin] RBAC: Adding query filter params (role.id=bffbe227cda1418cbb58394ddff6f861, scope.project.id=8e201e5f281a4977be503e22daf149d7, user.id=3ff152e22374407eb72caf648a763107) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:56.630216 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7c3f41-d2b4-4133-9e63-45d887eda38b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:56.631671 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7c3f41-d2b4-4133-9e63-45d887eda38b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:56.637316 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 647/1287] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 12:26:56 2022] GET /identity/v3/role_assignments?user.id=3ff152e22374407eb72caf648a763107&scope.project.id=8e201e5f281a4977be503e22daf149d7&role.id=bffbe227cda1418cbb58394ddff6f861 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:58.697039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd40e8-4500-4df6-90b5-3d8cb5fd9f6f None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:58.697557 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd40e8-4500-4df6-90b5-3d8cb5fd9f6f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:58.697557 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd40e8-4500-4df6-90b5-3d8cb5fd9f6f None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:58.698904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 641/1288] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:26:58.705242 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69bd683-7156-45ba-acad-fc44065f1d69 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:58.705764 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69bd683-7156-45ba-acad-fc44065f1d69 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:58.705764 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69bd683-7156-45ba-acad-fc44065f1d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:58.725626 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c69bd683-7156-45ba-acad-fc44065f1d69 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:26:58.726929 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c69bd683-7156-45ba-acad-fc44065f1d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:58.758469 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:58.797013 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 648/1289] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:26:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:26:58.819617 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd40e8-4500-4df6-90b5-3d8cb5fd9f6f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:58.924458 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd40e8-4500-4df6-90b5-3d8cb5fd9f6f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:58.935672 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:58.958377 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141853d8-24dd-48e7-ac57-b3f2eaa98e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:58.958946 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141853d8-24dd-48e7-ac57-b3f2eaa98e1d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:58.959252 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141853d8-24dd-48e7-ac57-b3f2eaa98e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:58.959528 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141853d8-24dd-48e7-ac57-b3f2eaa98e1d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:58.960100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141853d8-24dd-48e7-ac57-b3f2eaa98e1d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:58.961451 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141853d8-24dd-48e7-ac57-b3f2eaa98e1d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:58.969893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-141853d8-24dd-48e7-ac57-b3f2eaa98e1d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 12:26:58.969893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:58.969893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:26:58.969893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:26:58.969893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:26:58.969893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:26:58.969893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:26:58.969893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:26:58.969893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 12:26:58.969893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:26:58.969893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:26:58.969893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:26:58.969893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:26:58.969893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:26:58.969893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:26:58.969893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:26:58.969893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:26:58.971242 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:26:58.971242 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:26:58.971242 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:26:58.971242 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:26:58.971242 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:26:58.971242 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:26:58.971242 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:26:58.971242 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:26:58.971242 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:26:58.971242 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:26:58.971242 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:26:58.971242 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:26:58.971242 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:26:58.971242 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:26:58.971242 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:26:58.971242 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:26:58.972343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:26:58.972343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:26:58.972343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:26:58.972343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:26:58.972343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:26:58.972343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:26:58.972343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:26:58.972343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:26:58.972343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:26:58.972343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:26:58.972343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:26:58.972343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:26:58.972343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 12:26:58.972343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:26:58.972343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 642/1290] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 10 12:26:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:26:58.977298 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69bd683-7156-45ba-acad-fc44065f1d69 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:59.077307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69bd683-7156-45ba-acad-fc44065f1d69 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:59.111721 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8881f3-3221-4277-8fb4-a8b80ed5b724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:59.112317 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8881f3-3221-4277-8fb4-a8b80ed5b724 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:59.112617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8881f3-3221-4277-8fb4-a8b80ed5b724 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:59.112890 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8881f3-3221-4277-8fb4-a8b80ed5b724 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:59.113504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8881f3-3221-4277-8fb4-a8b80ed5b724 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:59.113804 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8881f3-3221-4277-8fb4-a8b80ed5b724 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:59.115147 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8881f3-3221-4277-8fb4-a8b80ed5b724 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:59.119814 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 649/1291] 10.222.0.49 () {56 vars in 1116 bytes} [Wed Aug 10 12:26:58 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:26:59.126599 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141853d8-24dd-48e7-ac57-b3f2eaa98e1d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:59.226231 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141853d8-24dd-48e7-ac57-b3f2eaa98e1d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:59.259531 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e2b47-5e5f-4503-befc-8bc30778c6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:59.260145 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e2b47-5e5f-4503-befc-8bc30778c6ad None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:59.260433 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e2b47-5e5f-4503-befc-8bc30778c6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:59.260708 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e2b47-5e5f-4503-befc-8bc30778c6ad None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:59.261334 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9e2b47-5e5f-4503-befc-8bc30778c6ad None admin] RBAC: Adding query filter params (type=placement) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:26:59.261639 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9e2b47-5e5f-4503-befc-8bc30778c6ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:59.263826 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9e2b47-5e5f-4503-befc-8bc30778c6ad None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:59.268422 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 643/1292] 10.222.0.49 () {56 vars in 1116 bytes} [Wed Aug 10 12:26:59 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:27:01.241392 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2531ee9-a93a-4594-a046-5d64dc778dbd None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:01.241968 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2531ee9-a93a-4594-a046-5d64dc778dbd None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:01.241968 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2531ee9-a93a-4594-a046-5d64dc778dbd None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:01.243554 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 650/1293] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:27:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:27:01.248761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c8bff-3462-4644-994d-31aeb54a6bba None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:01.249261 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c8bff-3462-4644-994d-31aeb54a6bba None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:01.249352 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c8bff-3462-4644-994d-31aeb54a6bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:01.267829 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-de4c8bff-3462-4644-994d-31aeb54a6bba None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:27:01.269124 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-de4c8bff-3462-4644-994d-31aeb54a6bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:27:01.299694 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:27:01.337077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 644/1294] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:27:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:27:01.360407 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2531ee9-a93a-4594-a046-5d64dc778dbd None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:27:01.392187 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:27:01.461389 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2531ee9-a93a-4594-a046-5d64dc778dbd None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:27:01.495056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181ea10-c84c-45ab-95d0-934bf92e55fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:27:01.495620 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181ea10-c84c-45ab-95d0-934bf92e55fc None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:01.495899 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181ea10-c84c-45ab-95d0-934bf92e55fc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:01.496174 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181ea10-c84c-45ab-95d0-934bf92e55fc None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:01.496813 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181ea10-c84c-45ab-95d0-934bf92e55fc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:27:01.498220 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181ea10-c84c-45ab-95d0-934bf92e55fc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:27:01.587960 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 651/1295] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 10 12:27:01 2022] POST /identity/v3/services => generated 246 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:27:03.581618 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef2cd5-8a0b-42db-8ce5-e116e6fae50b None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:03.582124 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef2cd5-8a0b-42db-8ce5-e116e6fae50b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:03.582124 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef2cd5-8a0b-42db-8ce5-e116e6fae50b None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:03.583466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 645/1296] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:27:03.588506 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680f10b-ea70-4b8f-a399-eb1ede04e608 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:03.588932 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680f10b-ea70-4b8f-a399-eb1ede04e608 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:03.589055 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680f10b-ea70-4b8f-a399-eb1ede04e608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:03.608442 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-2680f10b-ea70-4b8f-a399-eb1ede04e608 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:27:03.609623 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2680f10b-ea70-4b8f-a399-eb1ede04e608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:27:03.680158 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 652/1297] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:27:03 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:27:03.702771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef2cd5-8a0b-42db-8ce5-e116e6fae50b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:27:03.803319 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef2cd5-8a0b-42db-8ce5-e116e6fae50b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:27:03.837436 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ca1d3-4cb9-4ec9-a9f0-aa70fd52ae6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:27:03.838000 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ca1d3-4cb9-4ec9-a9f0-aa70fd52ae6f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:03.838280 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ca1d3-4cb9-4ec9-a9f0-aa70fd52ae6f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:03.838552 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ca1d3-4cb9-4ec9-a9f0-aa70fd52ae6f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:03.839122 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ca1d3-4cb9-4ec9-a9f0-aa70fd52ae6f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:27:03.840514 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ca1d3-4cb9-4ec9-a9f0-aa70fd52ae6f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:27:03.849575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-253ca1d3-4cb9-4ec9-a9f0-aa70fd52ae6f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 12:27:03.849575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:27:03.849575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:27:03.849575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:27:03.849575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:27:03.849575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:27:03.849575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:27:03.849575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:27:03.849575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 12:27:03.849575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:27:03.849575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:27:03.849575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:27:03.849575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:27:03.849575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:27:03.849575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:27:03.849575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:27:03.849575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:27:03.850743 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:27:03.850743 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:27:03.850743 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:27:03.850743 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:27:03.850743 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:27:03.850743 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:27:03.850743 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:27:03.850743 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:27:03.850743 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:27:03.850743 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:27:03.850743 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:27:03.850743 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:27:03.850743 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:27:03.850743 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:27:03.850743 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:27:03.850743 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:27:03.851828 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:27:03.851828 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:27:03.851828 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:27:03.851828 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:27:03.851828 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:27:03.851828 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:27:03.851828 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:27:03.851828 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:27:03.851828 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:27:03.851828 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:27:03.851828 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:27:03.851828 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:27:03.851828 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 12:27:03.851828 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:27:03.851828 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 646/1298] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 10 12:27:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:27:03.856917 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680f10b-ea70-4b8f-a399-eb1ede04e608 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:27:03.875077 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:27:03.954779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680f10b-ea70-4b8f-a399-eb1ede04e608 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:27:03.988060 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957e3b4-e57c-4a6d-b98c-f441e090a593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:27:03.988665 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957e3b4-e57c-4a6d-b98c-f441e090a593 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:03.988946 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957e3b4-e57c-4a6d-b98c-f441e090a593 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:03.989217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957e3b4-e57c-4a6d-b98c-f441e090a593 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:03.989832 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957e3b4-e57c-4a6d-b98c-f441e090a593 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:27:03.990132 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957e3b4-e57c-4a6d-b98c-f441e090a593 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:27:03.991511 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957e3b4-e57c-4a6d-b98c-f441e090a593 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:27:03.996483 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 653/1299] 10.222.0.49 () {56 vars in 1116 bytes} [Wed Aug 10 12:27:03 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:27:04.003853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ca1d3-4cb9-4ec9-a9f0-aa70fd52ae6f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:27:04.034428 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:27:04.099792 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ca1d3-4cb9-4ec9-a9f0-aa70fd52ae6f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:27:04.133980 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ceccc9-de8a-412a-8f69-d389782d6cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:27:04.134579 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ceccc9-de8a-412a-8f69-d389782d6cbb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:04.134856 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ceccc9-de8a-412a-8f69-d389782d6cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:04.135127 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ceccc9-de8a-412a-8f69-d389782d6cbb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:04.135836 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ceccc9-de8a-412a-8f69-d389782d6cbb None admin] RBAC: Adding query filter params (interface=public, service_id=7c6c52700b1049e799cc4b3940527126, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:27:04.136143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ceccc9-de8a-412a-8f69-d389782d6cbb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:27:04.137549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ceccc9-de8a-412a-8f69-d389782d6cbb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:27:04.143428 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 647/1300] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Aug 10 12:27:04 2022] GET /identity/v3/endpoints?service_id=7c6c52700b1049e799cc4b3940527126&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:27:04.149128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957e3b4-e57c-4a6d-b98c-f441e090a593 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:27:04.227256 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:27:04.244769 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957e3b4-e57c-4a6d-b98c-f441e090a593 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:27:04.279242 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d3887-6971-4e14-a1b6-aa61bc5417c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:27:04.279524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d3887-6971-4e14-a1b6-aa61bc5417c2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:04.279805 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d3887-6971-4e14-a1b6-aa61bc5417c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:04.280082 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d3887-6971-4e14-a1b6-aa61bc5417c2 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:04.280684 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d3887-6971-4e14-a1b6-aa61bc5417c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:27:04.282040 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d3887-6971-4e14-a1b6-aa61bc5417c2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:27:04.288851 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 654/1301] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Aug 10 12:27:04 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:27:06.281263 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c2409-2ed7-48c3-b6f1-afd88f20afb0 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:06.281685 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c2409-2ed7-48c3-b6f1-afd88f20afb0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:06.281800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c2409-2ed7-48c3-b6f1-afd88f20afb0 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:06.283138 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 648/1302] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:27:06.288177 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb1158-0436-4258-aa1d-eccb9b239848 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:06.288586 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb1158-0436-4258-aa1d-eccb9b239848 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:06.288732 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb1158-0436-4258-aa1d-eccb9b239848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:06.306365 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7bbb1158-0436-4258-aa1d-eccb9b239848 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:27:06.307587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7bbb1158-0436-4258-aa1d-eccb9b239848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:27:06.373878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 655/1303] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:27:06 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:27:06.398173 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573c2409-2ed7-48c3-b6f1-afd88f20afb0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:27:06.491892 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:27:06.498939 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573c2409-2ed7-48c3-b6f1-afd88f20afb0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:27:06.532524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d79e8d-33c0-4412-a270-12cb33e03056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:27:06.533085 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d79e8d-33c0-4412-a270-12cb33e03056 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:06.533368 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d79e8d-33c0-4412-a270-12cb33e03056 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:06.533644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d79e8d-33c0-4412-a270-12cb33e03056 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:06.534215 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d79e8d-33c0-4412-a270-12cb33e03056 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:27:06.535699 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d79e8d-33c0-4412-a270-12cb33e03056 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:27:06.544701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-57d79e8d-33c0-4412-a270-12cb33e03056 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 12:27:06.544701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:27:06.544701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:27:06.544701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:27:06.544701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:27:06.544701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:27:06.544701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:27:06.544701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:27:06.544701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 12:27:06.544701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:27:06.544701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:27:06.544701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:27:06.544701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:27:06.544701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:27:06.544701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:27:06.544701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:27:06.544701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:27:06.546444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:27:06.546444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:27:06.546444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:27:06.546444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:27:06.546444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:27:06.546444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:27:06.546444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:27:06.546444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:27:06.546444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:27:06.546444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:27:06.546444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:27:06.546444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:27:06.546444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:27:06.546444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:27:06.546444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:27:06.546444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:27:06.547372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:27:06.547372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Aug 10 12:27:06.547372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:27:06.547372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:27:06.547372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:27:06.547372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:27:06.547372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:27:06.547372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:27:06.547372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:27:06.547372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:27:06.547372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:27:06.547372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:27:06.547372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 12:27:06.547372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Aug 10 12:27:06.547372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 649/1304] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 10 12:27:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:27:06.551709 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb1158-0436-4258-aa1d-eccb9b239848 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:27:06.626766 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:27:06.648212 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb1158-0436-4258-aa1d-eccb9b239848 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:27:06.681248 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63c82d-0689-437a-97a2-428f99c27006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:27:06.681842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63c82d-0689-437a-97a2-428f99c27006 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:06.682114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63c82d-0689-437a-97a2-428f99c27006 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:06.682379 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63c82d-0689-437a-97a2-428f99c27006 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:06.683046 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63c82d-0689-437a-97a2-428f99c27006 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:27:06.683372 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63c82d-0689-437a-97a2-428f99c27006 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:27:06.685344 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63c82d-0689-437a-97a2-428f99c27006 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:27:06.690237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 656/1305] 10.222.0.49 () {56 vars in 1116 bytes} [Wed Aug 10 12:27:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:27:06.696251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d79e8d-33c0-4412-a270-12cb33e03056 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:27:06.796336 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d79e8d-33c0-4412-a270-12cb33e03056 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:27:06.830527 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858504de-5a77-40d9-bdfb-80a3fc83cff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:27:06.831127 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858504de-5a77-40d9-bdfb-80a3fc83cff4 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:06.831441 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858504de-5a77-40d9-bdfb-80a3fc83cff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:06.831716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858504de-5a77-40d9-bdfb-80a3fc83cff4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:06.832376 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858504de-5a77-40d9-bdfb-80a3fc83cff4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:27:06.833736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858504de-5a77-40d9-bdfb-80a3fc83cff4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:27:06.915392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 650/1306] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:27:06 2022] POST /identity/v3/endpoints => generated 331 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:27:16.721025 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=30 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:27:16.910849 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:18.219316 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ffcc8-3d5a-4500-b61c-24b09c13e640 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:18.219895 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ffcc8-3d5a-4500-b61c-24b09c13e640 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:18.219895 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ffcc8-3d5a-4500-b61c-24b09c13e640 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:18.226428 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da5636-bdb4-40a8-9654-e41f558eaeda None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:18.226870 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da5636-bdb4-40a8-9654-e41f558eaeda None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:18.226964 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da5636-bdb4-40a8-9654-e41f558eaeda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:18.231995 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 657/1307] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:33:18 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:33:18.306213 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-51da5636-bdb4-40a8-9654-e41f558eaeda None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:18.307442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-51da5636-bdb4-40a8-9654-e41f558eaeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:18.439663 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 651/1308] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:33:18.491982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16fd3f-593d-4504-963d-e7fa6f546d70 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:18.492460 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16fd3f-593d-4504-963d-e7fa6f546d70 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:18.493218 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16fd3f-593d-4504-963d-e7fa6f546d70 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:18.495885 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 658/1309] 10.222.0.49 () {54 vars in 965 bytes} [Wed Aug 10 12:33:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:33:18.503932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55b7c4-e267-4c26-bf2e-1343b6b1c41f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:18.504217 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55b7c4-e267-4c26-bf2e-1343b6b1c41f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:18.504487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55b7c4-e267-4c26-bf2e-1343b6b1c41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:18.628258 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0b55b7c4-e267-4c26-bf2e-1343b6b1c41f None None] MFA Rules not processed for user `5db814952f9f466abf6148b7a0db2173`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:18.630349 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0b55b7c4-e267-4c26-bf2e-1343b6b1c41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:18.658668 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:18.720368 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 652/1310] 10.222.0.49 () {58 vars in 1062 bytes} [Wed Aug 10 12:33:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:18.730185 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16fd3f-593d-4504-963d-e7fa6f546d70 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:18.732039 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bc16fd3f-593d-4504-963d-e7fa6f546d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:18.853625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16fd3f-593d-4504-963d-e7fa6f546d70 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:18.932234 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06534077-1ab5-40e5-b50a-184c1e49a813 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:18.932767 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06534077-1ab5-40e5-b50a-184c1e49a813 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:18.933048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06534077-1ab5-40e5-b50a-184c1e49a813 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:18.933322 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06534077-1ab5-40e5-b50a-184c1e49a813 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:18.935717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-06534077-1ab5-40e5-b50a-184c1e49a813 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:18.992993 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:19.049929 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06534077-1ab5-40e5-b50a-184c1e49a813 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:19.051395 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06534077-1ab5-40e5-b50a-184c1e49a813 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:19.070824 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 659/1311] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:33:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:33:21.390157 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78af7c9-0d9d-4588-a057-8097cb3cb3cd None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:21.390731 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78af7c9-0d9d-4588-a057-8097cb3cb3cd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:21.390731 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78af7c9-0d9d-4588-a057-8097cb3cb3cd None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:21.392240 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 653/1312] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:33:21.397296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581050b-2e62-4091-8fb9-7822bbc5a1c0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:21.397714 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581050b-2e62-4091-8fb9-7822bbc5a1c0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:21.397907 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581050b-2e62-4091-8fb9-7822bbc5a1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:21.481287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9581050b-2e62-4091-8fb9-7822bbc5a1c0 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:21.483061 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9581050b-2e62-4091-8fb9-7822bbc5a1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:21.591983 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 660/1313] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:21.631203 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78af7c9-0d9d-4588-a057-8097cb3cb3cd None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:21.633204 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c78af7c9-0d9d-4588-a057-8097cb3cb3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:21.761204 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78af7c9-0d9d-4588-a057-8097cb3cb3cd None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:21.853829 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:21.857188 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098ae21-d3eb-414f-b808-e282f63b074b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:21.857915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098ae21-d3eb-414f-b808-e282f63b074b service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:21.858201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098ae21-d3eb-414f-b808-e282f63b074b service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:21.858472 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098ae21-d3eb-414f-b808-e282f63b074b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:21.860826 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c098ae21-d3eb-414f-b808-e282f63b074b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:21.987409 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c098ae21-d3eb-414f-b808-e282f63b074b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:21.988968 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c098ae21-d3eb-414f-b808-e282f63b074b service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:22.009377 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 654/1314] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:33:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:33:24.608466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdc87b-c41f-45cc-a616-57bf73074cb0 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:24.609067 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdc87b-c41f-45cc-a616-57bf73074cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:24.609067 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdc87b-c41f-45cc-a616-57bf73074cb0 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:24.610642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 661/1315] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:33:24.615683 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e0e503-1e07-454b-9790-2b4f2fe9fc32 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:24.616124 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e0e503-1e07-454b-9790-2b4f2fe9fc32 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:24.616233 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e0e503-1e07-454b-9790-2b4f2fe9fc32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:24.680328 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-71e0e503-1e07-454b-9790-2b4f2fe9fc32 None None] MFA Rules not processed for user `a43f0c8b634b4f2997cbdf44c7dd2c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:24.681546 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-71e0e503-1e07-454b-9790-2b4f2fe9fc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:24.732862 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:24.775517 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 655/1316] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:33:24 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:24.818094 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdc87b-c41f-45cc-a616-57bf73074cb0 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:24.819719 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7bbdc87b-c41f-45cc-a616-57bf73074cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:24.829336 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:24.923067 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdc87b-c41f-45cc-a616-57bf73074cb0 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:24.994565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697aa002-9445-43b3-9954-e597f944784a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:24.995191 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697aa002-9445-43b3-9954-e597f944784a service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:24.995473 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697aa002-9445-43b3-9954-e597f944784a service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:24.995745 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697aa002-9445-43b3-9954-e597f944784a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:24.997404 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-697aa002-9445-43b3-9954-e597f944784a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:25.058149 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:25.103218 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697aa002-9445-43b3-9954-e597f944784a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:25.104645 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697aa002-9445-43b3-9954-e597f944784a service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:25.124052 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 662/1317] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:33:24 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:33:27.971785 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf96334-33e4-4d9b-91bf-c752f590def3 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:27.972342 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf96334-33e4-4d9b-91bf-c752f590def3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:27.972342 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf96334-33e4-4d9b-91bf-c752f590def3 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:27.973710 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 656/1318] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:33:27.978882 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbed072-89e9-4971-8d81-3b6a2ec6036b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:27.979337 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbed072-89e9-4971-8d81-3b6a2ec6036b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:27.979471 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbed072-89e9-4971-8d81-3b6a2ec6036b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:28.043663 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-cdbed072-89e9-4971-8d81-3b6a2ec6036b None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:28.045079 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cdbed072-89e9-4971-8d81-3b6a2ec6036b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:28.141451 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 663/1319] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:33:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:28.180804 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf96334-33e4-4d9b-91bf-c752f590def3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:28.182342 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ccf96334-33e4-4d9b-91bf-c752f590def3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:28.287300 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf96334-33e4-4d9b-91bf-c752f590def3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:28.359672 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3da14-e65c-40b2-81e1-19f7111a0699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:28.360279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3da14-e65c-40b2-81e1-19f7111a0699 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:28.360561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3da14-e65c-40b2-81e1-19f7111a0699 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:28.360839 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3da14-e65c-40b2-81e1-19f7111a0699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:28.362438 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b8c3da14-e65c-40b2-81e1-19f7111a0699 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:28.369681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:28.470176 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3da14-e65c-40b2-81e1-19f7111a0699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:28.471664 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3da14-e65c-40b2-81e1-19f7111a0699 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:28.490412 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 657/1320] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:33:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:33:31.457125 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a43036-a3bd-4da3-b15e-75ce9b61e748 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:31.457659 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a43036-a3bd-4da3-b15e-75ce9b61e748 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:31.457659 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a43036-a3bd-4da3-b15e-75ce9b61e748 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:31.459394 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 664/1321] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:33:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:33:31.464818 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840407a8-5ca1-4a3e-98fd-548a4853bbbe None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:31.465245 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840407a8-5ca1-4a3e-98fd-548a4853bbbe None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:31.465357 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840407a8-5ca1-4a3e-98fd-548a4853bbbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:31.537739 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-840407a8-5ca1-4a3e-98fd-548a4853bbbe None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:31.538897 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-840407a8-5ca1-4a3e-98fd-548a4853bbbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:31.602732 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:31.644478 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 658/1322] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:31.681436 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a43036-a3bd-4da3-b15e-75ce9b61e748 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:31.682984 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-23a43036-a3bd-4da3-b15e-75ce9b61e748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:31.704513 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:31.797823 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a43036-a3bd-4da3-b15e-75ce9b61e748 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:31.883492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d25297-342b-44fb-b662-1c3a0463f5f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:31.884308 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d25297-342b-44fb-b662-1c3a0463f5f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:31.884586 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d25297-342b-44fb-b662-1c3a0463f5f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:31.884857 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d25297-342b-44fb-b662-1c3a0463f5f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:31.886497 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-75d25297-342b-44fb-b662-1c3a0463f5f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:31.974198 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:32.019913 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d25297-342b-44fb-b662-1c3a0463f5f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:32.021377 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d25297-342b-44fb-b662-1c3a0463f5f0 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:32.043249 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 665/1323] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:33:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:33:34.110176 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6dd12-3f03-43a3-ae3b-2f962c1a6f99 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:34.110765 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6dd12-3f03-43a3-ae3b-2f962c1a6f99 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:34.110765 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6dd12-3f03-43a3-ae3b-2f962c1a6f99 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:34.112116 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 659/1324] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:33:34.118011 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06945d7-b0a9-4653-b933-b7b9dc088ae5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:34.118533 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06945d7-b0a9-4653-b933-b7b9dc088ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:34.118533 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06945d7-b0a9-4653-b933-b7b9dc088ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:34.195461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d06945d7-b0a9-4653-b933-b7b9dc088ae5 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:34.196706 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d06945d7-b0a9-4653-b933-b7b9dc088ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:34.301384 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 666/1325] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:34.337238 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6dd12-3f03-43a3-ae3b-2f962c1a6f99 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:34.338775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4ab6dd12-3f03-43a3-ae3b-2f962c1a6f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:34.445347 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6dd12-3f03-43a3-ae3b-2f962c1a6f99 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:34.518817 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a80702f-6593-436e-93c4-b649ab9d59d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:34.519460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80702f-6593-436e-93c4-b649ab9d59d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:34.519740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80702f-6593-436e-93c4-b649ab9d59d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:34.520008 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80702f-6593-436e-93c4-b649ab9d59d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:34.521610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0a80702f-6593-436e-93c4-b649ab9d59d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:34.538606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:34.630999 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a80702f-6593-436e-93c4-b649ab9d59d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:34.632493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a80702f-6593-436e-93c4-b649ab9d59d9 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:34.652561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 660/1326] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:33:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:33:36.742049 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0b854-6af2-4634-9544-d5d55d5d97f8 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:36.742560 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0b854-6af2-4634-9544-d5d55d5d97f8 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:36.742560 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0b854-6af2-4634-9544-d5d55d5d97f8 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:36.744102 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 667/1327] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:33:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:33:36.749702 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65290f3-b27c-4e7d-974d-936ce23e25a8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:36.750117 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65290f3-b27c-4e7d-974d-936ce23e25a8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:36.750291 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65290f3-b27c-4e7d-974d-936ce23e25a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:36.939186 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f65290f3-b27c-4e7d-974d-936ce23e25a8 None None] MFA Rules not processed for user `a43f0c8b634b4f2997cbdf44c7dd2c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:36.940430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f65290f3-b27c-4e7d-974d-936ce23e25a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:37.006267 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:37.041305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 661/1328] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:33:36 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:37.080335 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0b854-6af2-4634-9544-d5d55d5d97f8 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:37.081872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-13f0b854-6af2-4634-9544-d5d55d5d97f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:37.113087 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:37.186282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0b854-6af2-4634-9544-d5d55d5d97f8 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:37.261175 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3284bd1-fa3f-429b-8222-1f4753952d38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:37.261744 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3284bd1-fa3f-429b-8222-1f4753952d38 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:37.262052 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3284bd1-fa3f-429b-8222-1f4753952d38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:37.262329 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3284bd1-fa3f-429b-8222-1f4753952d38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:37.264531 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e3284bd1-fa3f-429b-8222-1f4753952d38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:37.347328 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:37.379715 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3284bd1-fa3f-429b-8222-1f4753952d38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:37.381236 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3284bd1-fa3f-429b-8222-1f4753952d38 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:37.400555 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 668/1329] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:33:37 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:33:39.850279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c702e6c-93f6-447a-8bba-14c87da0d6d5 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:39.850838 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c702e6c-93f6-447a-8bba-14c87da0d6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:39.850838 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c702e6c-93f6-447a-8bba-14c87da0d6d5 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:39.852209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 662/1330] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:33:39.858197 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030c0cb-994b-4456-8437-cc43d81ff3fa None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:39.858604 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030c0cb-994b-4456-8437-cc43d81ff3fa None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:39.858749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030c0cb-994b-4456-8437-cc43d81ff3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:39.924193 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a030c0cb-994b-4456-8437-cc43d81ff3fa None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:39.925407 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a030c0cb-994b-4456-8437-cc43d81ff3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:40.020211 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 669/1331] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:40.070089 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4fdb0-32d1-4bfc-afef-c30e4a0edda9 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:40.070372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4fdb0-32d1-4bfc-afef-c30e4a0edda9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:40.070646 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4fdb0-32d1-4bfc-afef-c30e4a0edda9 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:40.072100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 663/1332] 10.222.0.49 () {54 vars in 965 bytes} [Wed Aug 10 12:33:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:33:40.077681 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11f60b-c7dc-40ab-bd82-3a1a344fc614 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:40.077973 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11f60b-c7dc-40ab-bd82-3a1a344fc614 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:40.078253 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11f60b-c7dc-40ab-bd82-3a1a344fc614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:40.123671 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:40.154602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-4a11f60b-c7dc-40ab-bd82-3a1a344fc614 None None] MFA Rules not processed for user `5db814952f9f466abf6148b7a0db2173`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:40.155790 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4a11f60b-c7dc-40ab-bd82-3a1a344fc614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:40.235547 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 670/1333] 10.222.0.49 () {58 vars in 1062 bytes} [Wed Aug 10 12:33:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:40.244366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d4fdb0-32d1-4bfc-afef-c30e4a0edda9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:40.245900 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-21d4fdb0-32d1-4bfc-afef-c30e4a0edda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:40.355279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d4fdb0-32d1-4bfc-afef-c30e4a0edda9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:40.429512 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e7c68f-ea3c-4b67-9613-8274713017fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:40.430049 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7c68f-ea3c-4b67-9613-8274713017fe service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:40.430334 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7c68f-ea3c-4b67-9613-8274713017fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:40.430605 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7c68f-ea3c-4b67-9613-8274713017fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:40.432684 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-63e7c68f-ea3c-4b67-9613-8274713017fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:40.461246 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:40.542297 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e7c68f-ea3c-4b67-9613-8274713017fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:40.543872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e7c68f-ea3c-4b67-9613-8274713017fe service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:40.563261 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 664/1334] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:33:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:33:42.818868 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f79922-0961-4306-abe0-21f4e05e2ee8 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:42.819431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f79922-0961-4306-abe0-21f4e05e2ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:42.819431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f79922-0961-4306-abe0-21f4e05e2ee8 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:42.820780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 671/1335] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:33:42.825946 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ab25e-40f5-4658-9cc5-d9a661fc3159 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:42.826332 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ab25e-40f5-4658-9cc5-d9a661fc3159 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:42.826508 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ab25e-40f5-4658-9cc5-d9a661fc3159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:42.893267 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-131ab25e-40f5-4658-9cc5-d9a661fc3159 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:42.894496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-131ab25e-40f5-4658-9cc5-d9a661fc3159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:42.973932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:42.996968 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 665/1336] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:33:43.038363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f79922-0961-4306-abe0-21f4e05e2ee8 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:43.039947 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-29f79922-0961-4306-abe0-21f4e05e2ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:43.141716 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f79922-0961-4306-abe0-21f4e05e2ee8 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:43.169844 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:43.213183 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d621b-915f-4be5-83ff-166b8b12a5a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:43.213794 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d621b-915f-4be5-83ff-166b8b12a5a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:43.214072 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d621b-915f-4be5-83ff-166b8b12a5a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:43.214342 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d621b-915f-4be5-83ff-166b8b12a5a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:43.216601 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ea7d621b-915f-4be5-83ff-166b8b12a5a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:43.324237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7d621b-915f-4be5-83ff-166b8b12a5a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:43.326474 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7d621b-915f-4be5-83ff-166b8b12a5a4 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:43.344385 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 672/1337] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:33:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:33:46.261731 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0b275-fb4f-47bd-807d-716d0aebc725 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:46.262605 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0b275-fb4f-47bd-807d-716d0aebc725 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:46.263070 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0b275-fb4f-47bd-807d-716d0aebc725 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:46.264466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 666/1338] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:33:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:33:46.269626 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383c910-80ec-43f3-8536-a0b86688d2c6 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:46.270039 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383c910-80ec-43f3-8536-a0b86688d2c6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:46.270178 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383c910-80ec-43f3-8536-a0b86688d2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:46.335098 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3383c910-80ec-43f3-8536-a0b86688d2c6 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:46.336340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3383c910-80ec-43f3-8536-a0b86688d2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:46.345202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:46.432411 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 673/1339] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:33:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:46.469021 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0b275-fb4f-47bd-807d-716d0aebc725 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:46.470551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2bd0b275-fb4f-47bd-807d-716d0aebc725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:46.571774 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0b275-fb4f-47bd-807d-716d0aebc725 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:46.644704 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74afd34f-11d4-4d0e-8cd8-811c665e39c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:46.645308 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74afd34f-11d4-4d0e-8cd8-811c665e39c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:46.645589 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74afd34f-11d4-4d0e-8cd8-811c665e39c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:46.645863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74afd34f-11d4-4d0e-8cd8-811c665e39c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:46.647503 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-74afd34f-11d4-4d0e-8cd8-811c665e39c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:46.683074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:46.756226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74afd34f-11d4-4d0e-8cd8-811c665e39c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:46.757662 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74afd34f-11d4-4d0e-8cd8-811c665e39c1 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:46.776370 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 667/1340] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:33:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:33:51.135618 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a24f4-1e3f-4567-9871-91be82456add None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:51.136040 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a24f4-1e3f-4567-9871-91be82456add None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:51.136183 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a24f4-1e3f-4567-9871-91be82456add None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:51.137613 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 674/1341] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:33:51.143136 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db205d47-d0b2-48a1-86b9-f8ccec078fbc None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:51.143631 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db205d47-d0b2-48a1-86b9-f8ccec078fbc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:51.143728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db205d47-d0b2-48a1-86b9-f8ccec078fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:51.211308 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-db205d47-d0b2-48a1-86b9-f8ccec078fbc None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:51.212506 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-db205d47-d0b2-48a1-86b9-f8ccec078fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:51.297963 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:51.309892 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 668/1342] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:51.350478 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a24f4-1e3f-4567-9871-91be82456add None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:51.352017 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6c6a24f4-1e3f-4567-9871-91be82456add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:51.464504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a24f4-1e3f-4567-9871-91be82456add None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:51.502667 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:51.538610 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ffb39-fb6f-42e6-a424-2dd085e01a03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:51.539239 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ffb39-fb6f-42e6-a424-2dd085e01a03 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:51.539518 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ffb39-fb6f-42e6-a424-2dd085e01a03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:51.539785 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ffb39-fb6f-42e6-a424-2dd085e01a03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:51.541370 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f09ffb39-fb6f-42e6-a424-2dd085e01a03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:51.650271 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ffb39-fb6f-42e6-a424-2dd085e01a03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:51.651803 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ffb39-fb6f-42e6-a424-2dd085e01a03 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:51.670162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 675/1343] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:33:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:33:54.690835 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae014c-9efa-4d43-913b-a624605a36cb None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:54.691456 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae014c-9efa-4d43-913b-a624605a36cb None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:54.691456 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae014c-9efa-4d43-913b-a624605a36cb None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:54.692775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 669/1344] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:33:54.697897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba383a-0b43-4c70-860a-0230292aa279 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:54.698305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba383a-0b43-4c70-860a-0230292aa279 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:54.698449 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba383a-0b43-4c70-860a-0230292aa279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:54.765484 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a3ba383a-0b43-4c70-860a-0230292aa279 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:54.766705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a3ba383a-0b43-4c70-860a-0230292aa279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:54.785929 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:54.866740 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 676/1345] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:33:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:54.904631 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae014c-9efa-4d43-913b-a624605a36cb None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:54.906233 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f9ae014c-9efa-4d43-913b-a624605a36cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:55.011418 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae014c-9efa-4d43-913b-a624605a36cb None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:55.085872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c432fc-0091-4c7a-8ad1-3787be851b38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:55.086396 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c432fc-0091-4c7a-8ad1-3787be851b38 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:55.086675 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c432fc-0091-4c7a-8ad1-3787be851b38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:55.086947 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c432fc-0091-4c7a-8ad1-3787be851b38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:55.088603 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d7c432fc-0091-4c7a-8ad1-3787be851b38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:55.132223 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:55.202879 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c432fc-0091-4c7a-8ad1-3787be851b38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:55.204339 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c432fc-0091-4c7a-8ad1-3787be851b38 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:55.222532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 670/1346] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:33:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:33:59.394017 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7538b-153c-4851-80ba-41c95f00acc7 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:59.394572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7538b-153c-4851-80ba-41c95f00acc7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:59.394572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7538b-153c-4851-80ba-41c95f00acc7 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:59.395957 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 677/1347] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:33:59.401174 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4804b-b05e-4df4-9627-94fc7b605da3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:59.401599 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4804b-b05e-4df4-9627-94fc7b605da3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:59.401817 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4804b-b05e-4df4-9627-94fc7b605da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:59.474844 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-50b4804b-b05e-4df4-9627-94fc7b605da3 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:59.476109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-50b4804b-b05e-4df4-9627-94fc7b605da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:59.576527 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:59.579631 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 671/1348] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:59.619264 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7538b-153c-4851-80ba-41c95f00acc7 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:59.620827 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6cf7538b-153c-4851-80ba-41c95f00acc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:59.729880 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7538b-153c-4851-80ba-41c95f00acc7 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:59.779938 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:59.807999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e495516-7b3e-4558-8eb3-1153307bd490 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:59.808583 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e495516-7b3e-4558-8eb3-1153307bd490 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:59.809587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e495516-7b3e-4558-8eb3-1153307bd490 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:59.810099 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e495516-7b3e-4558-8eb3-1153307bd490 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:59.812396 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9e495516-7b3e-4558-8eb3-1153307bd490 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:59.923599 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e495516-7b3e-4558-8eb3-1153307bd490 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:59.924949 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e495516-7b3e-4558-8eb3-1153307bd490 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:59.945071 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 678/1349] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:33:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:34:04.883244 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b580fc6-abcb-4158-9f48-389a322bf1fd None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:04.883244 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b580fc6-abcb-4158-9f48-389a322bf1fd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:04.883244 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b580fc6-abcb-4158-9f48-389a322bf1fd None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:04.894038 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 672/1350] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 10 12:34:04 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:34:04.904970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e157f-cdb4-42cf-9051-519276c60ae0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:04.905465 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e157f-cdb4-42cf-9051-519276c60ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:04.906365 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e157f-cdb4-42cf-9051-519276c60ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:05.012927 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-920e157f-cdb4-42cf-9051-519276c60ae0 None None] MFA Rules not processed for user `3ff152e22374407eb72caf648a763107`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:34:05.014426 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-920e157f-cdb4-42cf-9051-519276c60ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:05.037485 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:05.138488 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 679/1351] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 10 12:34:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:34:09.342777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269be346-61bf-4455-8a08-e46add1ddd4c None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:09.343656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269be346-61bf-4455-8a08-e46add1ddd4c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:09.343930 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269be346-61bf-4455-8a08-e46add1ddd4c None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:09.345273 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 673/1352] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 10 12:34:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:34:09.350446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56649a66-cfac-4759-ac94-6953e4bcefd1 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:09.351319 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56649a66-cfac-4759-ac94-6953e4bcefd1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:09.351829 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56649a66-cfac-4759-ac94-6953e4bcefd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:09.440682 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-56649a66-cfac-4759-ac94-6953e4bcefd1 None None] MFA Rules not processed for user `3ff152e22374407eb72caf648a763107`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:34:09.442270 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-56649a66-cfac-4759-ac94-6953e4bcefd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:09.523145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:09.561215 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 680/1353] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 10 12:34:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:34:10.766172 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76f24b-4a33-4aab-bab5-f72c2ade1d7a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:10.766701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76f24b-4a33-4aab-bab5-f72c2ade1d7a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:10.766701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76f24b-4a33-4aab-bab5-f72c2ade1d7a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:10.768073 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 674/1354] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 10 12:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:34:10.773570 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013e412-de3f-482f-853f-99a7a52d0d89 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:10.779554 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013e412-de3f-482f-853f-99a7a52d0d89 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:10.779554 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013e412-de3f-482f-853f-99a7a52d0d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:10.853288 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7013e412-de3f-482f-853f-99a7a52d0d89 None None] MFA Rules not processed for user `3ff152e22374407eb72caf648a763107`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:34:10.854507 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7013e412-de3f-482f-853f-99a7a52d0d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:10.962992 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 681/1355] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 10 12:34:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:34:14.417112 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b741bf-367a-4baa-9b84-03fce1149e43 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:14.418629 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b741bf-367a-4baa-9b84-03fce1149e43 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:14.419063 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b741bf-367a-4baa-9b84-03fce1149e43 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:14.424678 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 675/1356] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:34:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:34:14.426212 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c160430-4af2-4179-af51-ab3fde574214 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:14.426629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c160430-4af2-4179-af51-ab3fde574214 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:14.427164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c160430-4af2-4179-af51-ab3fde574214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:14.467278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:14.501602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3c160430-4af2-4179-af51-ab3fde574214 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:34:14.503298 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3c160430-4af2-4179-af51-ab3fde574214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:14.656471 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 682/1357] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:34:14.708521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c0b9f-17f2-45b1-9641-61626f18454e None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:14.709777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c0b9f-17f2-45b1-9641-61626f18454e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:14.710225 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c0b9f-17f2-45b1-9641-61626f18454e None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:14.711762 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 676/1358] 10.222.0.49 () {54 vars in 962 bytes} [Wed Aug 10 12:34:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:34:14.733117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce212cb-ee25-4850-9984-f089c180ff64 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:14.733918 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce212cb-ee25-4850-9984-f089c180ff64 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:14.734371 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce212cb-ee25-4850-9984-f089c180ff64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:14.828524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5ce212cb-ee25-4850-9984-f089c180ff64 None None] MFA Rules not processed for user `eb3f6deffb54467f8b923956ed2a6acf`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:34:14.829941 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5ce212cb-ee25-4850-9984-f089c180ff64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:14.887460 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:14.965751 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 683/1359] 10.222.0.49 () {58 vars in 1059 bytes} [Wed Aug 10 12:34:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:34:14.973886 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c0b9f-17f2-45b1-9641-61626f18454e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:34:14.975543 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c42c0b9f-17f2-45b1-9641-61626f18454e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:15.128956 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c0b9f-17f2-45b1-9641-61626f18454e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:34:15.212281 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:15.267845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de2307-5a6f-4609-ab88-333cc76bbf6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:34:15.268447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de2307-5a6f-4609-ab88-333cc76bbf6a service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:15.268727 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de2307-5a6f-4609-ab88-333cc76bbf6a service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:15.269228 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de2307-5a6f-4609-ab88-333cc76bbf6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:15.270752 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-93de2307-5a6f-4609-ab88-333cc76bbf6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:15.370842 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de2307-5a6f-4609-ab88-333cc76bbf6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:34:15.373213 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de2307-5a6f-4609-ab88-333cc76bbf6a service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:34:15.392431 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 677/1360] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:34:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:34:17.634069 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bea28d-15c9-4ea7-9796-8421f1d6e7b6 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:17.634625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bea28d-15c9-4ea7-9796-8421f1d6e7b6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:17.634625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bea28d-15c9-4ea7-9796-8421f1d6e7b6 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:17.640617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 684/1361] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:34:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:34:17.642533 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d5806-0635-43b7-b7af-529f7552daf9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:17.643005 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d5806-0635-43b7-b7af-529f7552daf9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:17.643218 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d5806-0635-43b7-b7af-529f7552daf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:17.718277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-481d5806-0635-43b7-b7af-529f7552daf9 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:34:17.719587 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-481d5806-0635-43b7-b7af-529f7552daf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:17.766012 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:17.837457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 678/1362] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:34:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:34:17.886077 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bea28d-15c9-4ea7-9796-8421f1d6e7b6 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:34:17.888458 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-30bea28d-15c9-4ea7-9796-8421f1d6e7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:18.087210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bea28d-15c9-4ea7-9796-8421f1d6e7b6 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:34:18.097613 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:18.218501 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecf878-0112-4891-be3a-d85b502a487b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:34:18.219118 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecf878-0112-4891-be3a-d85b502a487b service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:18.219424 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecf878-0112-4891-be3a-d85b502a487b service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:18.219916 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecf878-0112-4891-be3a-d85b502a487b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:18.222069 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4eecf878-0112-4891-be3a-d85b502a487b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:18.326139 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eecf878-0112-4891-be3a-d85b502a487b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:34:18.327910 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eecf878-0112-4891-be3a-d85b502a487b service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:34:18.347449 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 685/1363] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:34:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:34:20.853681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd22bc-5509-4899-93f7-a72ded29c951 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:20.854291 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd22bc-5509-4899-93f7-a72ded29c951 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:20.854406 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd22bc-5509-4899-93f7-a72ded29c951 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:20.856698 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 679/1364] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:34:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:34:20.862266 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a808dfd-bfeb-4ef3-8be7-fe3bbb98a0c6 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:20.862691 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a808dfd-bfeb-4ef3-8be7-fe3bbb98a0c6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:20.862774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a808dfd-bfeb-4ef3-8be7-fe3bbb98a0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:20.878841 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:20.933075 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0a808dfd-bfeb-4ef3-8be7-fe3bbb98a0c6 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:34:20.935103 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0a808dfd-bfeb-4ef3-8be7-fe3bbb98a0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:21.038328 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 686/1365] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:34:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:34:21.085924 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926fde2-b672-4219-8367-fb65b8dd9c51 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:21.086365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926fde2-b672-4219-8367-fb65b8dd9c51 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:21.086718 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926fde2-b672-4219-8367-fb65b8dd9c51 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:21.088215 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 680/1366] 10.222.0.49 () {54 vars in 962 bytes} [Wed Aug 10 12:34:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:34:21.094122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c92bd-0d6b-4255-8f02-582d812de4fa None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:21.101977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c92bd-0d6b-4255-8f02-582d812de4fa None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:21.101977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c92bd-0d6b-4255-8f02-582d812de4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:21.183024 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a84c92bd-0d6b-4255-8f02-582d812de4fa None None] MFA Rules not processed for user `eb3f6deffb54467f8b923956ed2a6acf`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:34:21.185067 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a84c92bd-0d6b-4255-8f02-582d812de4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:21.222046 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:21.315983 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 687/1367] 10.222.0.49 () {58 vars in 1059 bytes} [Wed Aug 10 12:34:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:34:21.323782 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926fde2-b672-4219-8367-fb65b8dd9c51 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:34:21.325451 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3926fde2-b672-4219-8367-fb65b8dd9c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:21.485656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926fde2-b672-4219-8367-fb65b8dd9c51 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:34:21.506299 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:21.612126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6fe2b-a374-4e73-a8af-a137ccdc8c0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:34:21.612742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6fe2b-a374-4e73-a8af-a137ccdc8c0c service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:21.613016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6fe2b-a374-4e73-a8af-a137ccdc8c0c service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:21.613283 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6fe2b-a374-4e73-a8af-a137ccdc8c0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:21.614858 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bdc6fe2b-a374-4e73-a8af-a137ccdc8c0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:21.713315 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6fe2b-a374-4e73-a8af-a137ccdc8c0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:34:21.715509 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6fe2b-a374-4e73-a8af-a137ccdc8c0c service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:34:21.741458 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 681/1368] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:34:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:34:22.572347 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f135c-5026-4669-8dbd-da8478693b08 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:22.572865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f135c-5026-4669-8dbd-da8478693b08 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:22.572865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f135c-5026-4669-8dbd-da8478693b08 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:22.574327 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 688/1369] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:34:22.579342 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0997c51-171c-41c1-8e19-b0c82ee1454f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:22.579803 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0997c51-171c-41c1-8e19-b0c82ee1454f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:22.579887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0997c51-171c-41c1-8e19-b0c82ee1454f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:22.613143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:22.649490 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b0997c51-171c-41c1-8e19-b0c82ee1454f None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:34:22.651553 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b0997c51-171c-41c1-8e19-b0c82ee1454f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:22.756746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 682/1370] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:34:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:34:23.103974 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079a128-6862-4999-96c2-421aa62d399c None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:23.104254 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079a128-6862-4999-96c2-421aa62d399c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:23.104516 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079a128-6862-4999-96c2-421aa62d399c None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:23.106930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 689/1371] 10.222.0.49 () {54 vars in 964 bytes} [Wed Aug 10 12:34:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:34:23.112213 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ef3d4-4127-456f-8613-8708651e71d3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:23.112502 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ef3d4-4127-456f-8613-8708651e71d3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:23.114973 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ef3d4-4127-456f-8613-8708651e71d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:23.191916 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-874ef3d4-4127-456f-8613-8708651e71d3 None None] MFA Rules not processed for user `ab5a0d9c39d84cc7a91f5a752dcf2b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:34:23.193177 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-874ef3d4-4127-456f-8613-8708651e71d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:23.242410 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:23.283329 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 683/1372] 10.222.0.49 () {58 vars in 1061 bytes} [Wed Aug 10 12:34:23 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:34:23.288241 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079a128-6862-4999-96c2-421aa62d399c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:34:23.289874 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f079a128-6862-4999-96c2-421aa62d399c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:23.401780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079a128-6862-4999-96c2-421aa62d399c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:34:23.446139 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:23.483363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d17a5d-70df-4d5a-bdd7-a086affe10ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:34:23.484384 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d17a5d-70df-4d5a-bdd7-a086affe10ed service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:23.484763 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d17a5d-70df-4d5a-bdd7-a086affe10ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:23.485134 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d17a5d-70df-4d5a-bdd7-a086affe10ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:23.487234 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a2d17a5d-70df-4d5a-bdd7-a086affe10ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:23.610724 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d17a5d-70df-4d5a-bdd7-a086affe10ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:34:23.612228 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d17a5d-70df-4d5a-bdd7-a086affe10ed service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:34:23.631913 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 690/1373] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:34:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:34:23.994085 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5d4ee-77bd-4dd9-9e7d-6177c3f183d9 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:23.994542 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5d4ee-77bd-4dd9-9e7d-6177c3f183d9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:23.994775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5d4ee-77bd-4dd9-9e7d-6177c3f183d9 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:23.997253 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 684/1374] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:34:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:34:24.002800 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237e7e0-27b0-4113-9e81-82ac46eeede0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:24.003141 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237e7e0-27b0-4113-9e81-82ac46eeede0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:24.003703 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237e7e0-27b0-4113-9e81-82ac46eeede0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:24.104702 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7237e7e0-27b0-4113-9e81-82ac46eeede0 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:34:24.104702 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7237e7e0-27b0-4113-9e81-82ac46eeede0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:24.109354 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:24.187878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 691/1375] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:34:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:34:24.233924 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5d4ee-77bd-4dd9-9e7d-6177c3f183d9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:34:24.235492 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e9c5d4ee-77bd-4dd9-9e7d-6177c3f183d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:24.392336 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5d4ee-77bd-4dd9-9e7d-6177c3f183d9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:34:24.445050 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:24.508412 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6ae13-0aee-4b2c-bc6d-16f9d4e1446f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:34:24.509023 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6ae13-0aee-4b2c-bc6d-16f9d4e1446f service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:24.509305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6ae13-0aee-4b2c-bc6d-16f9d4e1446f service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:24.509582 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6ae13-0aee-4b2c-bc6d-16f9d4e1446f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:24.511783 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ebe6ae13-0aee-4b2c-bc6d-16f9d4e1446f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:24.603356 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6ae13-0aee-4b2c-bc6d-16f9d4e1446f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:34:24.604953 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6ae13-0aee-4b2c-bc6d-16f9d4e1446f service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:34:24.623763 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 685/1376] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:34:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:34:26.863134 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799d904-c4b0-4164-bf8f-ebc4ef61a0b7 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:26.863715 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799d904-c4b0-4164-bf8f-ebc4ef61a0b7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:26.863804 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799d904-c4b0-4164-bf8f-ebc4ef61a0b7 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:26.865229 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 692/1377] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:34:26.870377 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0149ee9-9139-4e26-bd15-b57bbdfa5321 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:26.871541 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0149ee9-9139-4e26-bd15-b57bbdfa5321 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:26.872017 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0149ee9-9139-4e26-bd15-b57bbdfa5321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:26.939206 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c0149ee9-9139-4e26-bd15-b57bbdfa5321 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:34:26.940416 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c0149ee9-9139-4e26-bd15-b57bbdfa5321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:26.947412 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:27.038958 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 686/1378] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:34:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:34:27.083244 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799d904-c4b0-4164-bf8f-ebc4ef61a0b7 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:34:27.084855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d799d904-c4b0-4164-bf8f-ebc4ef61a0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:27.282657 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799d904-c4b0-4164-bf8f-ebc4ef61a0b7 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:34:27.289767 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:27.542605 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c3172-29de-4258-ad13-38abf00d43b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:34:27.543304 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c3172-29de-4258-ad13-38abf00d43b6 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:27.543583 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c3172-29de-4258-ad13-38abf00d43b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:27.543851 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c3172-29de-4258-ad13-38abf00d43b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:27.545422 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-420c3172-29de-4258-ad13-38abf00d43b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:27.635467 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c3172-29de-4258-ad13-38abf00d43b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:34:27.638034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c3172-29de-4258-ad13-38abf00d43b6 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:34:27.657002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 693/1379] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:34:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:34:29.871533 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b28fb-f61e-4087-9d24-60611b00ab59 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:29.872077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b28fb-f61e-4087-9d24-60611b00ab59 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:29.872312 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b28fb-f61e-4087-9d24-60611b00ab59 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:29.873937 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 687/1380] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:34:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:34:29.879317 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d208d1a-ff2d-471b-bd24-12209e8f5e2f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:29.883711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d208d1a-ff2d-471b-bd24-12209e8f5e2f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:29.883711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d208d1a-ff2d-471b-bd24-12209e8f5e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:29.892779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:29.953933 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-2d208d1a-ff2d-471b-bd24-12209e8f5e2f None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:34:29.955304 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2d208d1a-ff2d-471b-bd24-12209e8f5e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:30.060381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 694/1381] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:34:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:34:30.107014 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b28fb-f61e-4087-9d24-60611b00ab59 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:34:30.109393 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-346b28fb-f61e-4087-9d24-60611b00ab59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:30.264156 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:30.283260 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b28fb-f61e-4087-9d24-60611b00ab59 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:34:30.405341 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843cd57e-16cb-495e-a97a-e62f1248bd0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:34:30.406100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843cd57e-16cb-495e-a97a-e62f1248bd0b service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:30.406482 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843cd57e-16cb-495e-a97a-e62f1248bd0b service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:30.406854 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843cd57e-16cb-495e-a97a-e62f1248bd0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:30.408568 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-843cd57e-16cb-495e-a97a-e62f1248bd0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:30.503452 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843cd57e-16cb-495e-a97a-e62f1248bd0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:34:30.505231 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843cd57e-16cb-495e-a97a-e62f1248bd0b service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:34:30.516955 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:30.530386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 688/1382] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:34:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:34:32.745106 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b37bb2-cacf-4b4a-999a-8d9b983212ec None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:32.746451 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b37bb2-cacf-4b4a-999a-8d9b983212ec None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:32.746451 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b37bb2-cacf-4b4a-999a-8d9b983212ec None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:32.747831 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 695/1383] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:34:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:34:32.753332 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd4bb7-fee6-4102-94a8-cf580db10e99 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:32.753812 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd4bb7-fee6-4102-94a8-cf580db10e99 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:32.753920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd4bb7-fee6-4102-94a8-cf580db10e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:32.824472 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b3bd4bb7-fee6-4102-94a8-cf580db10e99 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:34:32.825881 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b3bd4bb7-fee6-4102-94a8-cf580db10e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:32.928802 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 689/1384] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:34:32.973180 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b37bb2-cacf-4b4a-999a-8d9b983212ec None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:34:32.974882 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-11b37bb2-cacf-4b4a-999a-8d9b983212ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:33.067244 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:33.143404 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b37bb2-cacf-4b4a-999a-8d9b983212ec None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:34:33.266822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852dbb2-dbd1-4a0a-966e-fe39e9ba19a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:34:33.267462 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852dbb2-dbd1-4a0a-966e-fe39e9ba19a6 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:33.267737 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852dbb2-dbd1-4a0a-966e-fe39e9ba19a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:33.268015 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852dbb2-dbd1-4a0a-966e-fe39e9ba19a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:33.269631 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0852dbb2-dbd1-4a0a-966e-fe39e9ba19a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:33.325133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:33.366547 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852dbb2-dbd1-4a0a-966e-fe39e9ba19a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:34:33.368213 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852dbb2-dbd1-4a0a-966e-fe39e9ba19a6 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:34:33.389629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 696/1385] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:34:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:34:35.740276 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe88d1-37fc-43f4-bd04-2feea623f6fa None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:35.741042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe88d1-37fc-43f4-bd04-2feea623f6fa None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:35.741042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe88d1-37fc-43f4-bd04-2feea623f6fa None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:35.742586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 690/1386] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:34:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:34:35.748829 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cba93a-fe70-412e-9188-3a8750b390c3 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:35.749301 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cba93a-fe70-412e-9188-3a8750b390c3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:35.749398 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cba93a-fe70-412e-9188-3a8750b390c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:35.818988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-99cba93a-fe70-412e-9188-3a8750b390c3 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:34:35.820252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-99cba93a-fe70-412e-9188-3a8750b390c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:35.920971 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 697/1387] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:34:35.969236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe88d1-37fc-43f4-bd04-2feea623f6fa None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:34:35.970879 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-76fe88d1-37fc-43f4-bd04-2feea623f6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:36.048870 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:36.121044 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe88d1-37fc-43f4-bd04-2feea623f6fa None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:34:36.250518 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a9b02-dae3-4112-ab50-2af8b4094207 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:34:36.251319 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a9b02-dae3-4112-ab50-2af8b4094207 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:36.251773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a9b02-dae3-4112-ab50-2af8b4094207 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:36.252176 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a9b02-dae3-4112-ab50-2af8b4094207 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:36.254198 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f61a9b02-dae3-4112-ab50-2af8b4094207 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:36.299416 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:36.354551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61a9b02-dae3-4112-ab50-2af8b4094207 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:34:36.356265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61a9b02-dae3-4112-ab50-2af8b4094207 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:34:36.375491 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 691/1388] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:34:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:34:38.570113 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154e986-4400-4bed-82db-ba8e38c82d02 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:38.570653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154e986-4400-4bed-82db-ba8e38c82d02 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:38.570653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154e986-4400-4bed-82db-ba8e38c82d02 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:38.572047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 698/1389] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:34:38.578061 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6f1a9-c0ab-4acf-8282-7f43905759f8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:38.578497 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6f1a9-c0ab-4acf-8282-7f43905759f8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:38.578593 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6f1a9-c0ab-4acf-8282-7f43905759f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:38.646055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-feb6f1a9-c0ab-4acf-8282-7f43905759f8 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:34:38.647303 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-feb6f1a9-c0ab-4acf-8282-7f43905759f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:38.747830 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 692/1390] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:34:38.792585 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154e986-4400-4bed-82db-ba8e38c82d02 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:34:38.794300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3154e986-4400-4bed-82db-ba8e38c82d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:38.826233 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:38.940945 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154e986-4400-4bed-82db-ba8e38c82d02 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:34:39.043304 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:39.054593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a44271-55c6-421e-b1e1-6faa0449f307 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:34:39.055269 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a44271-55c6-421e-b1e1-6faa0449f307 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:39.055551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a44271-55c6-421e-b1e1-6faa0449f307 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:39.055826 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a44271-55c6-421e-b1e1-6faa0449f307 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:39.057391 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-88a44271-55c6-421e-b1e1-6faa0449f307 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:39.148655 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a44271-55c6-421e-b1e1-6faa0449f307 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:34:39.150156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a44271-55c6-421e-b1e1-6faa0449f307 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:34:39.169457 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 699/1391] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:34:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:34:39.601185 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639491c-25ee-44b5-a750-c747d011f322 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:39.601722 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639491c-25ee-44b5-a750-c747d011f322 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:39.601839 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639491c-25ee-44b5-a750-c747d011f322 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:39.603233 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 693/1392] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:34:39.609529 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bbd29-666b-4e06-8aea-a9f7d90f464c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:39.609951 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bbd29-666b-4e06-8aea-a9f7d90f464c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:39.610058 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bbd29-666b-4e06-8aea-a9f7d90f464c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:39.677214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-fe0bbd29-666b-4e06-8aea-a9f7d90f464c None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:34:39.678460 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-fe0bbd29-666b-4e06-8aea-a9f7d90f464c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:39.737871 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:39.775097 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 700/1393] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:34:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:34:39.831284 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77345f2-985e-4232-b8ec-27d141da3339 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:39.831667 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77345f2-985e-4232-b8ec-27d141da3339 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:39.832004 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77345f2-985e-4232-b8ec-27d141da3339 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:39.833457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 694/1394] 10.222.0.49 () {54 vars in 967 bytes} [Wed Aug 10 12:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:34:39.838726 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce18413-cad8-43f3-a6be-9dd868fe4ff4 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:39.839009 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce18413-cad8-43f3-a6be-9dd868fe4ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:39.839832 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce18413-cad8-43f3-a6be-9dd868fe4ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:39.916034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-bce18413-cad8-43f3-a6be-9dd868fe4ff4 None None] MFA Rules not processed for user `35dd27fd356241b885706375d880222f`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:34:39.917171 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bce18413-cad8-43f3-a6be-9dd868fe4ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:39.993143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 701/1395] 10.222.0.49 () {58 vars in 1064 bytes} [Wed Aug 10 12:34:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:34:40.001307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77345f2-985e-4232-b8ec-27d141da3339 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:34:40.002852 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a77345f2-985e-4232-b8ec-27d141da3339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:40.017090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:40.109231 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77345f2-985e-4232-b8ec-27d141da3339 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:34:40.181531 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b9eee-cb24-4063-8f7f-b90618d9a5da service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35dd27fd356241b885706375d880222f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:34:40.182129 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b9eee-cb24-4063-8f7f-b90618d9a5da service glance] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:40.182407 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b9eee-cb24-4063-8f7f-b90618d9a5da service glance] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:40.182684 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b9eee-cb24-4063-8f7f-b90618d9a5da service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:40.184430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c32b9eee-cb24-4063-8f7f-b90618d9a5da service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:40.251886 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:40.294410 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b9eee-cb24-4063-8f7f-b90618d9a5da service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:34:40.295887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b9eee-cb24-4063-8f7f-b90618d9a5da service glance] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:34:40.314971 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 695/1396] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:34:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:34:41.207000 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc532f0-07b7-4ebb-a874-caacde88af08 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:41.207503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc532f0-07b7-4ebb-a874-caacde88af08 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:41.207662 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc532f0-07b7-4ebb-a874-caacde88af08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:41.281979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6bc532f0-07b7-4ebb-a874-caacde88af08 None None] MFA Rules not processed for user `e42059e9af9842548ea08dcd39ff0b6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:34:41.283234 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6bc532f0-07b7-4ebb-a874-caacde88af08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:41.291736 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:41.364165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48185a-88fa-4ad7-a3f2-41067e8de7c5 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:41.364711 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48185a-88fa-4ad7-a3f2-41067e8de7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:41.364711 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48185a-88fa-4ad7-a3f2-41067e8de7c5 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:41.366071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 696/1397] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:34:41.371426 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 702/1398] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 10 12:34:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:34:41.371830 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ec03d-5ca6-4411-9cf6-92d71abf4ebe None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:41.372117 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ec03d-5ca6-4411-9cf6-92d71abf4ebe None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:41.372392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ec03d-5ca6-4411-9cf6-92d71abf4ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:41.415626 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42d887-e6f7-4576-b899-1b91fd684929 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:41.415993 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42d887-e6f7-4576-b899-1b91fd684929 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:41.416596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42d887-e6f7-4576-b899-1b91fd684929 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:41.418291 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 703/1399] 10.222.0.49 () {54 vars in 943 bytes} [Wed Aug 10 12:34:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:34:41.424339 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c78c68-d718-435c-a576-dc985295a345 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:41.424625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c78c68-d718-435c-a576-dc985295a345 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:41.424906 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c78c68-d718-435c-a576-dc985295a345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:41.443781 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1f8ec03d-5ca6-4411-9cf6-92d71abf4ebe None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:34:41.444941 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1f8ec03d-5ca6-4411-9cf6-92d71abf4ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:41.500520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-43c78c68-d718-435c-a576-dc985295a345 None None] MFA Rules not processed for user `59395bfa505c4540b3957deee78eb698`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:34:41.501906 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-43c78c68-d718-435c-a576-dc985295a345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:41.545777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 697/1400] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:34:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:34:41.588089 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:41.590547 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ec03d-5ca6-4411-9cf6-92d71abf4ebe None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:34:41.591860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1f8ec03d-5ca6-4411-9cf6-92d71abf4ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:41.613227 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 704/1401] 10.222.0.49 () {58 vars in 1040 bytes} [Wed Aug 10 12:34:41 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:34:41.618912 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:41.623044 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c78c68-d718-435c-a576-dc985295a345 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:34:41.633005 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c78c68-d718-435c-a576-dc985295a345 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:34:41.709200 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ec03d-5ca6-4411-9cf6-92d71abf4ebe None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:34:41.739723 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16157170-4a4e-4719-b706-99d5401d76b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59395bfa505c4540b3957deee78eb698', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:34:41.740339 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16157170-4a4e-4719-b706-99d5401d76b6 service swift] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:41.740623 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16157170-4a4e-4719-b706-99d5401d76b6 service swift] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:41.740964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16157170-4a4e-4719-b706-99d5401d76b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:41.745781 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-16157170-4a4e-4719-b706-99d5401d76b6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:41.829247 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16157170-4a4e-4719-b706-99d5401d76b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:34:41.830371 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b3f6a-3fa9-48b6-bfc8-1e38aeb3225a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:34:41.830877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16157170-4a4e-4719-b706-99d5401d76b6 service swift] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:34:41.831014 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b3f6a-3fa9-48b6-bfc8-1e38aeb3225a service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:41.831287 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b3f6a-3fa9-48b6-bfc8-1e38aeb3225a service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:41.831563 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b3f6a-3fa9-48b6-bfc8-1e38aeb3225a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:41.833171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-114b3f6a-3fa9-48b6-bfc8-1e38aeb3225a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:41.838887 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 705/1402] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 12:34:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 12:34:41.851141 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:41.924327 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b3f6a-3fa9-48b6-bfc8-1e38aeb3225a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:34:41.925928 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b3f6a-3fa9-48b6-bfc8-1e38aeb3225a service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:34:41.944642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 698/1403] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:34:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:34:44.175478 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a2a0c-ab93-4493-b9ab-d104696d0a71 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:44.176031 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a2a0c-ab93-4493-b9ab-d104696d0a71 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:44.176031 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a2a0c-ab93-4493-b9ab-d104696d0a71 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:44.177386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 706/1404] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:34:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:34:44.182620 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341dce5-8d31-47bc-89e4-5cdfb48b518f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:44.183069 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341dce5-8d31-47bc-89e4-5cdfb48b518f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:44.183900 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341dce5-8d31-47bc-89e4-5cdfb48b518f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:44.262945 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e341dce5-8d31-47bc-89e4-5cdfb48b518f None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:34:44.264204 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e341dce5-8d31-47bc-89e4-5cdfb48b518f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:44.371106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:44.399501 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 699/1405] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:34:44.425470 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a2a0c-ab93-4493-b9ab-d104696d0a71 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:34:44.427140 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-110a2a0c-ab93-4493-b9ab-d104696d0a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:44.438102 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:44.598299 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a2a0c-ab93-4493-b9ab-d104696d0a71 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:34:44.701188 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:44.740135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f1e75-9b05-4ce7-835a-aa0d87626bc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:34:44.740773 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f1e75-9b05-4ce7-835a-aa0d87626bc2 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:44.741048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f1e75-9b05-4ce7-835a-aa0d87626bc2 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:44.741316 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f1e75-9b05-4ce7-835a-aa0d87626bc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:44.742913 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cb9f1e75-9b05-4ce7-835a-aa0d87626bc2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:44.868812 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9f1e75-9b05-4ce7-835a-aa0d87626bc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:34:44.870988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9f1e75-9b05-4ce7-835a-aa0d87626bc2 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:34:44.893787 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 707/1406] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:34:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:34:47.911631 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e6a96-1d6b-41bd-a779-e8de119b287a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:47.912208 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e6a96-1d6b-41bd-a779-e8de119b287a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:47.912208 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e6a96-1d6b-41bd-a779-e8de119b287a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:47.913534 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 700/1407] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:34:47.919216 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c29e9-95c6-48ff-a041-37e77a91271e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:47.919216 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c29e9-95c6-48ff-a041-37e77a91271e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:47.919786 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c29e9-95c6-48ff-a041-37e77a91271e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:47.996206 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-244c29e9-95c6-48ff-a041-37e77a91271e None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:34:47.997699 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-244c29e9-95c6-48ff-a041-37e77a91271e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:48.047987 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:48.106743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 708/1408] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:34:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:34:48.151271 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e6a96-1d6b-41bd-a779-e8de119b287a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:34:48.153492 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3f5e6a96-1d6b-41bd-a779-e8de119b287a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:48.325776 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e6a96-1d6b-41bd-a779-e8de119b287a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:34:48.402151 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:48.454616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26784f-31bf-459e-aa06-386c883a75ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:34:48.455312 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26784f-31bf-459e-aa06-386c883a75ad service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:48.455599 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26784f-31bf-459e-aa06-386c883a75ad service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:48.455880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26784f-31bf-459e-aa06-386c883a75ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:48.457709 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ae26784f-31bf-459e-aa06-386c883a75ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:48.561265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26784f-31bf-459e-aa06-386c883a75ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:34:48.562782 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26784f-31bf-459e-aa06-386c883a75ad service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:34:48.581718 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 701/1409] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:34:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:34:58.104252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=23 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:58.581646 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=64 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:04.775037 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582598ee-7edd-4289-b78f-72054c4cf353 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:04.775660 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582598ee-7edd-4289-b78f-72054c4cf353 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:04.775660 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582598ee-7edd-4289-b78f-72054c4cf353 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:04.777682 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 709/1410] 10.222.0.49 () {54 vars in 918 bytes} [Wed Aug 10 12:35:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:35:04.782876 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4be31c-e130-4512-837b-ef9a76058e07 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:04.783346 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4be31c-e130-4512-837b-ef9a76058e07 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:04.783456 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4be31c-e130-4512-837b-ef9a76058e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:04.859393 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4e4be31c-e130-4512-837b-ef9a76058e07 None None] MFA Rules not processed for user `3ff152e22374407eb72caf648a763107`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:35:04.860613 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4e4be31c-e130-4512-837b-ef9a76058e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:04.967341 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 702/1411] 10.222.0.49 () {58 vars in 1015 bytes} [Wed Aug 10 12:35:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:35:04.982140 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b3dfd-3ae3-4e3d-804c-8ebd0eb993b0 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:04.982413 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b3dfd-3ae3-4e3d-804c-8ebd0eb993b0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:04.982691 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b3dfd-3ae3-4e3d-804c-8ebd0eb993b0 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:04.984340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 710/1412] 10.222.0.49 () {54 vars in 918 bytes} [Wed Aug 10 12:35:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:35:04.988808 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38711f5-e960-4b0b-83df-508f1455b2de None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:04.989093 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38711f5-e960-4b0b-83df-508f1455b2de None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:04.989367 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38711f5-e960-4b0b-83df-508f1455b2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:05.063769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:05.066318 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d38711f5-e960-4b0b-83df-508f1455b2de None None] MFA Rules not processed for user `eb3f6deffb54467f8b923956ed2a6acf`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:35:05.067593 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d38711f5-e960-4b0b-83df-508f1455b2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:05.185033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 703/1413] 10.222.0.49 () {58 vars in 1015 bytes} [Wed Aug 10 12:35:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:35:07.655345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d9640-d437-4010-9159-96063e9b0ea7 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:07.655869 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d9640-d437-4010-9159-96063e9b0ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:07.655869 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d9640-d437-4010-9159-96063e9b0ea7 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:07.657176 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:35:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:35:07.662310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad74be-523a-4ce9-aefa-2e5bc53d4843 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:07.662771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad74be-523a-4ce9-aefa-2e5bc53d4843 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:07.662884 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad74be-523a-4ce9-aefa-2e5bc53d4843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:07.728201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-16ad74be-523a-4ce9-aefa-2e5bc53d4843 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:35:07.729489 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-16ad74be-523a-4ce9-aefa-2e5bc53d4843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:07.783969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:07.831051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:35:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:35:07.875016 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d9640-d437-4010-9159-96063e9b0ea7 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:35:07.876596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-185d9640-d437-4010-9159-96063e9b0ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:08.020637 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d9640-d437-4010-9159-96063e9b0ea7 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:35:08.092938 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:08.132235 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c9bdb-ff8c-405a-9006-be62bbc435eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:35:08.132835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c9bdb-ff8c-405a-9006-be62bbc435eb service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:08.133108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c9bdb-ff8c-405a-9006-be62bbc435eb service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:08.133373 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c9bdb-ff8c-405a-9006-be62bbc435eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:08.134958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-991c9bdb-ff8c-405a-9006-be62bbc435eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:08.222499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991c9bdb-ff8c-405a-9006-be62bbc435eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:35:08.224106 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991c9bdb-ff8c-405a-9006-be62bbc435eb service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:35:08.242397 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 712/1416] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:35:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:35:09.143268 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a92fa8-5b5d-4fb7-91ca-471128159058 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:09.143832 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a92fa8-5b5d-4fb7-91ca-471128159058 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:09.143832 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a92fa8-5b5d-4fb7-91ca-471128159058 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:09.145175 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 12:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:35:09.150280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b55cb-ad8a-48d5-a13d-eb6cd81cc1f7 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:09.150744 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b55cb-ad8a-48d5-a13d-eb6cd81cc1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:09.150850 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b55cb-ad8a-48d5-a13d-eb6cd81cc1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:09.226132 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-518b55cb-ad8a-48d5-a13d-eb6cd81cc1f7 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:35:09.227386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-518b55cb-ad8a-48d5-a13d-eb6cd81cc1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:09.253830 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:09.322916 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 12:35:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:35:09.472725 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a92fa8-5b5d-4fb7-91ca-471128159058 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:35:09.474177 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-76a92fa8-5b5d-4fb7-91ca-471128159058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:09.618748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a92fa8-5b5d-4fb7-91ca-471128159058 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:35:09.658852 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:09.731498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74073c90-78d3-4e7d-ad10-f8970304225a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:35:09.732037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74073c90-78d3-4e7d-ad10-f8970304225a service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:09.732317 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74073c90-78d3-4e7d-ad10-f8970304225a service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:09.732587 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74073c90-78d3-4e7d-ad10-f8970304225a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:09.734546 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-74073c90-78d3-4e7d-ad10-f8970304225a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:09.825206 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74073c90-78d3-4e7d-ad10-f8970304225a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:35:09.826918 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74073c90-78d3-4e7d-ad10-f8970304225a service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:35:09.845339 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 706/1419] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:35:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:35:10.721204 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154167c-f85d-44c6-8424-7311be7d7105 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:10.721776 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154167c-f85d-44c6-8424-7311be7d7105 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:10.721776 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154167c-f85d-44c6-8424-7311be7d7105 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:10.723101 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 12:35:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:35:10.727876 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc798eb0-3b98-4512-a09f-28152c36b0bd None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:10.728307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc798eb0-3b98-4512-a09f-28152c36b0bd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:10.728454 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc798eb0-3b98-4512-a09f-28152c36b0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:10.783739 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:10.805765 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-fc798eb0-3b98-4512-a09f-28152c36b0bd None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:35:10.807022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fc798eb0-3b98-4512-a09f-28152c36b0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:10.905855 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 12:35:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:35:11.058476 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154167c-f85d-44c6-8424-7311be7d7105 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:35:11.060041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3154167c-f85d-44c6-8424-7311be7d7105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:11.206603 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154167c-f85d-44c6-8424-7311be7d7105 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:35:11.218759 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:11.322074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8651d74-0c31-4a2f-8aa0-6dd41ec13235 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:35:11.322629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8651d74-0c31-4a2f-8aa0-6dd41ec13235 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:11.322896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8651d74-0c31-4a2f-8aa0-6dd41ec13235 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:11.323896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8651d74-0c31-4a2f-8aa0-6dd41ec13235 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:11.325498 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b8651d74-0c31-4a2f-8aa0-6dd41ec13235 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:11.414670 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8651d74-0c31-4a2f-8aa0-6dd41ec13235 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:35:11.416283 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8651d74-0c31-4a2f-8aa0-6dd41ec13235 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:35:11.434537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 715/1422] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:35:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:35:12.292928 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc19b86-0978-468f-8bd8-0c9e7754b5d9 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:12.293389 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc19b86-0978-468f-8bd8-0c9e7754b5d9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:12.293487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc19b86-0978-468f-8bd8-0c9e7754b5d9 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:12.294807 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 12:35:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:35:12.299796 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77800e38-56e2-439a-96e2-b0e1de698459 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:12.300241 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77800e38-56e2-439a-96e2-b0e1de698459 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:12.300360 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77800e38-56e2-439a-96e2-b0e1de698459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:12.320757 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:12.376202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-77800e38-56e2-439a-96e2-b0e1de698459 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:35:12.377430 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-77800e38-56e2-439a-96e2-b0e1de698459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:12.471930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 12:35:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:35:12.624057 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc19b86-0978-468f-8bd8-0c9e7754b5d9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:35:12.625571 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0dc19b86-0978-468f-8bd8-0c9e7754b5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:12.735680 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:12.770547 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc19b86-0978-468f-8bd8-0c9e7754b5d9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:35:12.882330 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d3db8-5039-40b7-a7f6-a01389f010f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:35:12.882930 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d3db8-5039-40b7-a7f6-a01389f010f1 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:12.883236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d3db8-5039-40b7-a7f6-a01389f010f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:12.883507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d3db8-5039-40b7-a7f6-a01389f010f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:12.885096 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ed6d3db8-5039-40b7-a7f6-a01389f010f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:12.958798 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:12.972495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d3db8-5039-40b7-a7f6-a01389f010f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:35:12.974077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d3db8-5039-40b7-a7f6-a01389f010f1 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:35:12.992758 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 709/1425] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:35:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:35:13.894939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae739a2-6cfe-4fea-84f2-b6cc27691f1a None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:13.895489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae739a2-6cfe-4fea-84f2-b6cc27691f1a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:13.895489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae739a2-6cfe-4fea-84f2-b6cc27691f1a None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:13.896916 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 12:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:35:13.902172 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cec6da-5fed-45ca-9be2-3e394cce1123 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:13.902618 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cec6da-5fed-45ca-9be2-3e394cce1123 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:13.902718 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cec6da-5fed-45ca-9be2-3e394cce1123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:13.978904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-87cec6da-5fed-45ca-9be2-3e394cce1123 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:35:13.980699 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-87cec6da-5fed-45ca-9be2-3e394cce1123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:14.076807 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 12:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:35:14.229915 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae739a2-6cfe-4fea-84f2-b6cc27691f1a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:35:14.231468 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3ae739a2-6cfe-4fea-84f2-b6cc27691f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:14.317298 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:14.379550 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae739a2-6cfe-4fea-84f2-b6cc27691f1a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:35:14.491436 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d156bde-ea28-456b-a100-f51594cc15fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:35:14.491967 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d156bde-ea28-456b-a100-f51594cc15fa service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:14.492242 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d156bde-ea28-456b-a100-f51594cc15fa service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:14.492508 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d156bde-ea28-456b-a100-f51594cc15fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:14.494095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3d156bde-ea28-456b-a100-f51594cc15fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:14.540752 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:14.581371 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d156bde-ea28-456b-a100-f51594cc15fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:35:14.582951 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d156bde-ea28-456b-a100-f51594cc15fa service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:35:14.601170 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 718/1428] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:35:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:35:15.468161 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545f740-119e-43a3-ad09-e3e5d39dcecf None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:15.468721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545f740-119e-43a3-ad09-e3e5d39dcecf None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:15.468721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545f740-119e-43a3-ad09-e3e5d39dcecf None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:15.469960 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 12:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:35:15.474775 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5b6be-0553-45cd-a38d-56aca7e6efcc None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:15.475230 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5b6be-0553-45cd-a38d-56aca7e6efcc None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:15.475373 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5b6be-0553-45cd-a38d-56aca7e6efcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:15.550630 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-1fd5b6be-0553-45cd-a38d-56aca7e6efcc None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:35:15.551880 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1fd5b6be-0553-45cd-a38d-56aca7e6efcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:15.648464 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 12:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:35:15.800815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545f740-119e-43a3-ad09-e3e5d39dcecf None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:35:15.802343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9545f740-119e-43a3-ad09-e3e5d39dcecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:15.856606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:15.952568 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545f740-119e-43a3-ad09-e3e5d39dcecf None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:35:16.061890 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de81fb1-a3c9-41e3-83e8-6584fdf8a3b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:35:16.062410 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de81fb1-a3c9-41e3-83e8-6584fdf8a3b5 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:16.062691 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de81fb1-a3c9-41e3-83e8-6584fdf8a3b5 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:16.062960 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de81fb1-a3c9-41e3-83e8-6584fdf8a3b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:16.064584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6de81fb1-a3c9-41e3-83e8-6584fdf8a3b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:16.071324 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:16.154748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de81fb1-a3c9-41e3-83e8-6584fdf8a3b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:35:16.156386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de81fb1-a3c9-41e3-83e8-6584fdf8a3b5 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:35:16.174683 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 712/1431] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:35:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:35:17.056290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631974a3-941b-42ab-9f7c-c1a796ebe225 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:17.056885 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631974a3-941b-42ab-9f7c-c1a796ebe225 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:17.056885 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631974a3-941b-42ab-9f7c-c1a796ebe225 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:17.058364 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 12:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:35:17.063429 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46262d-8b4a-4718-8d62-23d49c9610b1 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:17.063915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46262d-8b4a-4718-8d62-23d49c9610b1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:17.064033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46262d-8b4a-4718-8d62-23d49c9610b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:17.139905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8f46262d-8b4a-4718-8d62-23d49c9610b1 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:35:17.141121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8f46262d-8b4a-4718-8d62-23d49c9610b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:17.209050 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:17.238578 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 12:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:35:17.392103 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631974a3-941b-42ab-9f7c-c1a796ebe225 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:35:17.393565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-631974a3-941b-42ab-9f7c-c1a796ebe225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:17.410685 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:17.538089 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631974a3-941b-42ab-9f7c-c1a796ebe225 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:35:17.625245 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:17.652788 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140a54e-3d05-443a-a9d9-914b4ca6aaa0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:35:17.653401 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140a54e-3d05-443a-a9d9-914b4ca6aaa0 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:17.653692 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140a54e-3d05-443a-a9d9-914b4ca6aaa0 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:17.653967 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140a54e-3d05-443a-a9d9-914b4ca6aaa0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:17.655691 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8140a54e-3d05-443a-a9d9-914b4ca6aaa0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:17.745302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8140a54e-3d05-443a-a9d9-914b4ca6aaa0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:35:17.746948 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8140a54e-3d05-443a-a9d9-914b4ca6aaa0 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:35:17.766059 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 721/1434] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:35:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:35:18.632184 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b9a74-cf45-467b-98e3-9a44d544ccea None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:18.632731 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b9a74-cf45-467b-98e3-9a44d544ccea None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:18.632731 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b9a74-cf45-467b-98e3-9a44d544ccea None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:18.634062 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 12:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:35:18.639854 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de04a11-7d01-40c9-a41a-90b244b53de9 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:18.640306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de04a11-7d01-40c9-a41a-90b244b53de9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:18.640425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de04a11-7d01-40c9-a41a-90b244b53de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:18.717773 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-2de04a11-7d01-40c9-a41a-90b244b53de9 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:35:18.718979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2de04a11-7d01-40c9-a41a-90b244b53de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:18.761051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:18.816359 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 12:35:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:35:18.967499 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65b9a74-cf45-467b-98e3-9a44d544ccea None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:35:18.969121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a65b9a74-cf45-467b-98e3-9a44d544ccea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:19.115015 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65b9a74-cf45-467b-98e3-9a44d544ccea None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:35:19.168322 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:19.226475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d962883-8cec-40d6-aa23-208eea53b3b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:35:19.227010 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d962883-8cec-40d6-aa23-208eea53b3b2 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:19.227321 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d962883-8cec-40d6-aa23-208eea53b3b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:19.227595 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d962883-8cec-40d6-aa23-208eea53b3b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:19.229817 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0d962883-8cec-40d6-aa23-208eea53b3b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:19.320780 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d962883-8cec-40d6-aa23-208eea53b3b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:35:19.322360 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d962883-8cec-40d6-aa23-208eea53b3b2 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:35:19.340527 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 715/1437] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:35:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:35:20.192852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1496166-4090-463f-8e2b-d3134a7ba660 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:20.193397 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1496166-4090-463f-8e2b-d3134a7ba660 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:20.193397 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1496166-4090-463f-8e2b-d3134a7ba660 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:20.194677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 12:35:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:35:20.199767 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9a9bd-815f-423b-8a9a-16ebecb8424c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:20.200178 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9a9bd-815f-423b-8a9a-16ebecb8424c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:20.200315 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9a9bd-815f-423b-8a9a-16ebecb8424c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:20.275095 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-94d9a9bd-815f-423b-8a9a-16ebecb8424c None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:35:20.276839 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-94d9a9bd-815f-423b-8a9a-16ebecb8424c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:20.278889 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:20.374146 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 12:35:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:35:20.529286 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1496166-4090-463f-8e2b-d3134a7ba660 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:35:20.530805 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f1496166-4090-463f-8e2b-d3134a7ba660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:20.676393 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1496166-4090-463f-8e2b-d3134a7ba660 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:35:20.702557 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:20.788202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6883419-eb02-4579-b40f-670fd02f1366 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:35:20.788728 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6883419-eb02-4579-b40f-670fd02f1366 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:20.789007 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6883419-eb02-4579-b40f-670fd02f1366 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:20.789279 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6883419-eb02-4579-b40f-670fd02f1366 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:20.790884 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f6883419-eb02-4579-b40f-670fd02f1366 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:20.880682 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6883419-eb02-4579-b40f-670fd02f1366 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:35:20.882251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6883419-eb02-4579-b40f-670fd02f1366 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:35:20.900853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 724/1440] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:35:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:35:22.834610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12345b66-f4cb-4b0b-8ea9-ea44eda90268 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:22.835215 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12345b66-f4cb-4b0b-8ea9-ea44eda90268 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:22.835215 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12345b66-f4cb-4b0b-8ea9-ea44eda90268 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:22.836551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:35:22.842420 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3f3fe-900f-462e-8679-a8463dd47b92 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:22.843139 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3f3fe-900f-462e-8679-a8463dd47b92 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:22.843352 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3f3fe-900f-462e-8679-a8463dd47b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:22.883973 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:22.909611 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c1b3f3fe-900f-462e-8679-a8463dd47b92 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:35:22.910822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c1b3f3fe-900f-462e-8679-a8463dd47b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:23.006530 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:35:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:35:23.045816 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12345b66-f4cb-4b0b-8ea9-ea44eda90268 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:35:23.047425 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-12345b66-f4cb-4b0b-8ea9-ea44eda90268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:23.151839 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12345b66-f4cb-4b0b-8ea9-ea44eda90268 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:35:23.184133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:23.225462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab182b7e-5756-4963-9b14-eb60ccdd2058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:35:23.226061 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab182b7e-5756-4963-9b14-eb60ccdd2058 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:23.226337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab182b7e-5756-4963-9b14-eb60ccdd2058 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:23.226604 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab182b7e-5756-4963-9b14-eb60ccdd2058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:23.228220 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ab182b7e-5756-4963-9b14-eb60ccdd2058 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:23.335970 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab182b7e-5756-4963-9b14-eb60ccdd2058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:35:23.337409 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab182b7e-5756-4963-9b14-eb60ccdd2058 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:35:23.356772 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 718/1443] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:35:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:35:25.590908 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc5653-65ae-49cd-9445-3f6a2962b72b None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:25.591615 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc5653-65ae-49cd-9445-3f6a2962b72b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:25.591615 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc5653-65ae-49cd-9445-3f6a2962b72b None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:25.593258 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:35:25.598541 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ba743-3e3d-4531-961c-d7df31e94317 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:25.598985 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ba743-3e3d-4531-961c-d7df31e94317 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:25.599095 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ba743-3e3d-4531-961c-d7df31e94317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:25.665202 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c18ba743-3e3d-4531-961c-d7df31e94317 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:35:25.666410 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c18ba743-3e3d-4531-961c-d7df31e94317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:25.679650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:25.763532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:35:25.802168 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc5653-65ae-49cd-9445-3f6a2962b72b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:35:25.803835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0dbc5653-65ae-49cd-9445-3f6a2962b72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:25.908448 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc5653-65ae-49cd-9445-3f6a2962b72b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:35:25.920758 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:25.981576 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad891434-6a24-46b1-b45d-12935ac7f82b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:35:25.982183 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad891434-6a24-46b1-b45d-12935ac7f82b service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:25.982466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad891434-6a24-46b1-b45d-12935ac7f82b service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:25.982736 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad891434-6a24-46b1-b45d-12935ac7f82b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:25.984623 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ad891434-6a24-46b1-b45d-12935ac7f82b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:26.089903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad891434-6a24-46b1-b45d-12935ac7f82b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:35:26.091378 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad891434-6a24-46b1-b45d-12935ac7f82b service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:35:26.109241 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 727/1446] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:35:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:35:28.505227 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d2a63-651c-450c-84c6-0affdf18aa04 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:28.505741 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d2a63-651c-450c-84c6-0affdf18aa04 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:28.505878 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d2a63-651c-450c-84c6-0affdf18aa04 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:28.507131 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:35:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:35:28.512435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e12e30-e257-491c-b0f7-b5a2269e0512 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:28.512811 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e12e30-e257-491c-b0f7-b5a2269e0512 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:28.512980 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e12e30-e257-491c-b0f7-b5a2269e0512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:28.560701 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:28.578660 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-73e12e30-e257-491c-b0f7-b5a2269e0512 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:35:28.583296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-73e12e30-e257-491c-b0f7-b5a2269e0512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:28.676560 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:35:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:35:28.715695 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d2a63-651c-450c-84c6-0affdf18aa04 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:35:28.717267 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-691d2a63-651c-450c-84c6-0affdf18aa04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:28.820011 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d2a63-651c-450c-84c6-0affdf18aa04 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:35:28.861025 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:28.895148 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5b6e4-1311-4598-880d-d2d0b442c08d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:35:28.895781 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5b6e4-1311-4598-880d-d2d0b442c08d service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:28.896059 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5b6e4-1311-4598-880d-d2d0b442c08d service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:28.896327 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5b6e4-1311-4598-880d-d2d0b442c08d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:28.897917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c8f5b6e4-1311-4598-880d-d2d0b442c08d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:29.005154 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5b6e4-1311-4598-880d-d2d0b442c08d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:35:29.006619 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5b6e4-1311-4598-880d-d2d0b442c08d service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:35:29.025221 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 721/1449] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:35:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:35:31.155339 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e832ffb-31be-4c6e-816f-910ef300aea4 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:31.155933 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e832ffb-31be-4c6e-816f-910ef300aea4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:31.155933 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e832ffb-31be-4c6e-816f-910ef300aea4 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:31.157257 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:35:31.162907 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b0048-af00-4a7f-a6d3-ef964a593e35 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:31.163366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b0048-af00-4a7f-a6d3-ef964a593e35 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:31.163493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b0048-af00-4a7f-a6d3-ef964a593e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:31.229462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-317b0048-af00-4a7f-a6d3-ef964a593e35 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:35:31.230682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-317b0048-af00-4a7f-a6d3-ef964a593e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:31.251221 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:31.329553 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:35:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:35:31.368261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e832ffb-31be-4c6e-816f-910ef300aea4 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:35:31.370408 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8e832ffb-31be-4c6e-816f-910ef300aea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:31.472890 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e832ffb-31be-4c6e-816f-910ef300aea4 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:35:31.492456 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:31.548317 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b392a6-3c8d-4e13-9bca-2c4206bd4fde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:35:31.548926 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b392a6-3c8d-4e13-9bca-2c4206bd4fde service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:31.549208 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b392a6-3c8d-4e13-9bca-2c4206bd4fde service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:31.549481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b392a6-3c8d-4e13-9bca-2c4206bd4fde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:31.551096 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a9b392a6-3c8d-4e13-9bca-2c4206bd4fde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:31.658831 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b392a6-3c8d-4e13-9bca-2c4206bd4fde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:35:31.660297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b392a6-3c8d-4e13-9bca-2c4206bd4fde service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:35:31.679444 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 730/1452] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:35:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:35:33.755248 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b7219-a961-4354-ad00-ec6fab518938 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:33.755769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b7219-a961-4354-ad00-ec6fab518938 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:33.755869 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b7219-a961-4354-ad00-ec6fab518938 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:33.757208 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:35:33.763328 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b439de9-1d7f-41ff-9255-7e9553a64c9f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:33.763724 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b439de9-1d7f-41ff-9255-7e9553a64c9f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:33.763878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b439de9-1d7f-41ff-9255-7e9553a64c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:33.828285 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:33.830999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5b439de9-1d7f-41ff-9255-7e9553a64c9f None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:35:33.832272 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5b439de9-1d7f-41ff-9255-7e9553a64c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:33.930308 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:35:33.968560 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b7219-a961-4354-ad00-ec6fab518938 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:35:33.970168 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1e7b7219-a961-4354-ad00-ec6fab518938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:34.073277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b7219-a961-4354-ad00-ec6fab518938 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:35:34.123450 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:34.147888 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e5502-0f6a-401a-abe3-af3c498082f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:35:34.148561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e5502-0f6a-401a-abe3-af3c498082f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:34.148954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e5502-0f6a-401a-abe3-af3c498082f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:34.149235 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e5502-0f6a-401a-abe3-af3c498082f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:34.150881 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-158e5502-0f6a-401a-abe3-af3c498082f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:34.261589 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e5502-0f6a-401a-abe3-af3c498082f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:35:34.263028 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e5502-0f6a-401a-abe3-af3c498082f3 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:35:34.282890 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 724/1455] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:35:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:35:36.406944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95051d-fcdf-4f27-b096-8560d667948a None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:36.408463 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95051d-fcdf-4f27-b096-8560d667948a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:36.408463 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95051d-fcdf-4f27-b096-8560d667948a None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:36.408989 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:35:36.414114 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2fb9f-c42c-4dc8-b2db-0f7a9cb7c274 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:36.414522 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2fb9f-c42c-4dc8-b2db-0f7a9cb7c274 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:36.414663 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2fb9f-c42c-4dc8-b2db-0f7a9cb7c274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:36.480957 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9fc2fb9f-c42c-4dc8-b2db-0f7a9cb7c274 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:35:36.482060 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9fc2fb9f-c42c-4dc8-b2db-0f7a9cb7c274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:36.511450 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:36.579503 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:35:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:35:36.619502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95051d-fcdf-4f27-b096-8560d667948a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:35:36.621124 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-af95051d-fcdf-4f27-b096-8560d667948a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:36.724986 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95051d-fcdf-4f27-b096-8560d667948a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:35:36.754531 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:36.800624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654bf77-966c-4be0-90e8-33e67b8a4561 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:35:36.801228 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654bf77-966c-4be0-90e8-33e67b8a4561 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:36.801508 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654bf77-966c-4be0-90e8-33e67b8a4561 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:36.801776 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654bf77-966c-4be0-90e8-33e67b8a4561 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:36.803392 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9654bf77-966c-4be0-90e8-33e67b8a4561 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:36.913157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9654bf77-966c-4be0-90e8-33e67b8a4561 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:35:36.914576 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9654bf77-966c-4be0-90e8-33e67b8a4561 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:35:36.934745 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 733/1458] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:35:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:35:39.003764 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81373c7d-19e3-4873-ac49-637e8491a2a4 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:39.004342 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81373c7d-19e3-4873-ac49-637e8491a2a4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:39.004342 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81373c7d-19e3-4873-ac49-637e8491a2a4 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:39.005662 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:35:39.011513 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48c918-3757-49f9-a161-883fc1549de7 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:39.011921 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48c918-3757-49f9-a161-883fc1549de7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:39.012056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48c918-3757-49f9-a161-883fc1549de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:39.076813 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-1d48c918-3757-49f9-a161-883fc1549de7 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:35:39.078022 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1d48c918-3757-49f9-a161-883fc1549de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:39.086181 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:39.173659 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:35:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:35:39.210658 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81373c7d-19e3-4873-ac49-637e8491a2a4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:35:39.212255 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-81373c7d-19e3-4873-ac49-637e8491a2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:39.315560 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81373c7d-19e3-4873-ac49-637e8491a2a4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:35:39.373156 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:39.391115 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b307fd-9da5-4418-8aed-62a00bd01e30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:35:39.391749 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b307fd-9da5-4418-8aed-62a00bd01e30 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:39.392028 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b307fd-9da5-4418-8aed-62a00bd01e30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:39.392299 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b307fd-9da5-4418-8aed-62a00bd01e30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:39.393885 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-95b307fd-9da5-4418-8aed-62a00bd01e30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:39.501291 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b307fd-9da5-4418-8aed-62a00bd01e30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:35:39.502721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b307fd-9da5-4418-8aed-62a00bd01e30 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:35:39.521343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 727/1461] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:35:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:35:41.568100 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9a551-84b4-4571-b02e-e7d744c7fcb7 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:41.568627 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9a551-84b4-4571-b02e-e7d744c7fcb7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:41.568627 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9a551-84b4-4571-b02e-e7d744c7fcb7 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:41.569988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:35:41.575842 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58691ae-7c97-46f2-b631-a72b76a1e992 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:41.576277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58691ae-7c97-46f2-b631-a72b76a1e992 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:41.576381 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58691ae-7c97-46f2-b631-a72b76a1e992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:41.641848 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d58691ae-7c97-46f2-b631-a72b76a1e992 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:35:41.643063 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d58691ae-7c97-46f2-b631-a72b76a1e992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:41.682425 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:41.741493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:35:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:35:41.779722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a9a551-84b4-4571-b02e-e7d744c7fcb7 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:35:41.781301 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d9a9a551-84b4-4571-b02e-e7d744c7fcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:41.883945 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a9a551-84b4-4571-b02e-e7d744c7fcb7 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:35:41.920873 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:41.956044 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a6709-6006-49bf-9f10-da619531ca50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:35:41.956653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a6709-6006-49bf-9f10-da619531ca50 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:41.956934 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a6709-6006-49bf-9f10-da619531ca50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:41.957208 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a6709-6006-49bf-9f10-da619531ca50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:41.958812 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-894a6709-6006-49bf-9f10-da619531ca50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:42.068306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894a6709-6006-49bf-9f10-da619531ca50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:35:42.069732 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894a6709-6006-49bf-9f10-da619531ca50 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:35:42.088007 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 736/1464] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:35:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:35:44.163204 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb434e06-28d5-462f-9007-d1ce2400ab19 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:44.163740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb434e06-28d5-462f-9007-d1ce2400ab19 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:44.163740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb434e06-28d5-462f-9007-d1ce2400ab19 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:44.165022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:35:44.171125 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057e993-e1d8-4bfa-80ed-3dda686d96f8 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:44.171605 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057e993-e1d8-4bfa-80ed-3dda686d96f8 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:44.171735 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057e993-e1d8-4bfa-80ed-3dda686d96f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:44.238175 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c057e993-e1d8-4bfa-80ed-3dda686d96f8 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:35:44.239425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c057e993-e1d8-4bfa-80ed-3dda686d96f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:44.259093 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:44.337312 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:35:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:35:44.374493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb434e06-28d5-462f-9007-d1ce2400ab19 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:35:44.376178 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cb434e06-28d5-462f-9007-d1ce2400ab19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:44.478547 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb434e06-28d5-462f-9007-d1ce2400ab19 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:35:44.544593 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:44.551156 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56e937-fa48-4e10-981d-963639300771 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:35:44.554725 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56e937-fa48-4e10-981d-963639300771 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:44.554725 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56e937-fa48-4e10-981d-963639300771 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:44.554725 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56e937-fa48-4e10-981d-963639300771 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:44.554725 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5e56e937-fa48-4e10-981d-963639300771 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:44.664388 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e56e937-fa48-4e10-981d-963639300771 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:35:44.665823 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e56e937-fa48-4e10-981d-963639300771 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:35:44.684128 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 730/1467] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:35:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:35:46.765055 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3397c-b368-426b-aa85-1c72e79199fb None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:46.765618 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3397c-b368-426b-aa85-1c72e79199fb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:46.765618 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3397c-b368-426b-aa85-1c72e79199fb None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:46.767029 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:35:46.772425 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7479fc0-79c9-4c01-a339-a0593e015e04 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:46.772867 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7479fc0-79c9-4c01-a339-a0593e015e04 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:46.772964 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7479fc0-79c9-4c01-a339-a0593e015e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:46.838822 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f7479fc0-79c9-4c01-a339-a0593e015e04 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:35:46.840075 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f7479fc0-79c9-4c01-a339-a0593e015e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:46.888909 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:46.937023 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:35:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:35:46.974214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3397c-b368-426b-aa85-1c72e79199fb None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:35:46.975805 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1af3397c-b368-426b-aa85-1c72e79199fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:47.079853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3397c-b368-426b-aa85-1c72e79199fb None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:35:47.131824 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:47.158525 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6564b4-be30-41f7-bcfd-8bb7fd2de8d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:35:47.159255 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6564b4-be30-41f7-bcfd-8bb7fd2de8d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:47.159506 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6564b4-be30-41f7-bcfd-8bb7fd2de8d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:47.159779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6564b4-be30-41f7-bcfd-8bb7fd2de8d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:47.161378 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2e6564b4-be30-41f7-bcfd-8bb7fd2de8d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:47.268367 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6564b4-be30-41f7-bcfd-8bb7fd2de8d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:35:47.269808 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6564b4-be30-41f7-bcfd-8bb7fd2de8d8 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:35:47.289737 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 739/1470] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:35:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:35:56.935265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=22 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:57.287444 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=66 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:36:07.625339 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7ae00-7e10-44f6-92c3-d8d956a0e1c1 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:07.625862 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7ae00-7e10-44f6-92c3-d8d956a0e1c1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:07.625862 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7ae00-7e10-44f6-92c3-d8d956a0e1c1 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:07.627153 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 732/1471] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:36:07.632399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729dad5e-34d4-40f6-a67a-44c58e2ffd59 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:07.632817 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729dad5e-34d4-40f6-a67a-44c58e2ffd59 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:07.632964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729dad5e-34d4-40f6-a67a-44c58e2ffd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:07.697885 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-729dad5e-34d4-40f6-a67a-44c58e2ffd59 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:36:07.699205 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-729dad5e-34d4-40f6-a67a-44c58e2ffd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:07.794936 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 740/1472] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:36:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:36:07.879796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea813d-7123-454b-b277-4b5455d05c8a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:07.880025 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea813d-7123-454b-b277-4b5455d05c8a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:07.880300 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea813d-7123-454b-b277-4b5455d05c8a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:07.881618 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 733/1473] 10.222.0.49 () {54 vars in 967 bytes} [Wed Aug 10 12:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:36:07.887804 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21e27a-855f-49eb-abfe-b63399ebda55 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:07.888116 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21e27a-855f-49eb-abfe-b63399ebda55 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:07.888394 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21e27a-855f-49eb-abfe-b63399ebda55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:07.964185 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-df21e27a-855f-49eb-abfe-b63399ebda55 None None] MFA Rules not processed for user `35dd27fd356241b885706375d880222f`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:36:07.965402 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-df21e27a-855f-49eb-abfe-b63399ebda55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:07.986525 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:36:08.042892 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 741/1474] 10.222.0.49 () {58 vars in 1064 bytes} [Wed Aug 10 12:36:07 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:36:08.050663 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea813d-7123-454b-b277-4b5455d05c8a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:08.052276 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c4ea813d-7123-454b-b277-4b5455d05c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:08.161856 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea813d-7123-454b-b277-4b5455d05c8a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:08.234663 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bfabdd-b483-4673-9a36-05a1ace51864 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35dd27fd356241b885706375d880222f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:08.235304 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfabdd-b483-4673-9a36-05a1ace51864 service glance] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:08.235588 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfabdd-b483-4673-9a36-05a1ace51864 service glance] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:08.235862 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfabdd-b483-4673-9a36-05a1ace51864 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:08.237453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-02bfabdd-b483-4673-9a36-05a1ace51864 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:08.294237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:36:08.348910 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bfabdd-b483-4673-9a36-05a1ace51864 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:08.350377 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bfabdd-b483-4673-9a36-05a1ace51864 service glance] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:08.370479 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 734/1475] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:36:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:36:10.471659 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8ed8a-fd34-4a11-b9a2-1d3fc06351d1 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:10.472223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8ed8a-fd34-4a11-b9a2-1d3fc06351d1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:10.472223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8ed8a-fd34-4a11-b9a2-1d3fc06351d1 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:10.473602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 742/1476] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:36:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:36:10.478902 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a365243-1f2e-471b-b944-82058bef2a24 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:10.479367 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a365243-1f2e-471b-b944-82058bef2a24 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:10.479486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a365243-1f2e-471b-b944-82058bef2a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:10.545880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1a365243-1f2e-471b-b944-82058bef2a24 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:36:10.547095 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1a365243-1f2e-471b-b944-82058bef2a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:10.644000 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 735/1477] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:36:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:36:10.667236 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8ed8a-fd34-4a11-b9a2-1d3fc06351d1 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:10.668776 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0ec8ed8a-fd34-4a11-b9a2-1d3fc06351d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:10.794558 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8ed8a-fd34-4a11-b9a2-1d3fc06351d1 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:10.845703 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:36:10.892452 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c48d079-c591-4654-afc7-970a13e1c142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:10.893415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48d079-c591-4654-afc7-970a13e1c142 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:10.893681 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48d079-c591-4654-afc7-970a13e1c142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:10.893947 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48d079-c591-4654-afc7-970a13e1c142 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:10.894579 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c48d079-c591-4654-afc7-970a13e1c142 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:10.896601 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: WARNING py.warnings [None req-1c48d079-c591-4654-afc7-970a13e1c142 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:36:10.896601 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: warnings.warn(msg) Aug 10 12:36:10.896601 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: Aug 10 12:36:10.897739 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c48d079-c591-4654-afc7-970a13e1c142 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:10.917058 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 743/1478] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 12:36:10 2022] GET /identity/v3/projects => generated 2553 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:36:12.960811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19cea17-3b00-4bf3-ad76-637b95496087 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:12.961411 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19cea17-3b00-4bf3-ad76-637b95496087 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:12.961411 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19cea17-3b00-4bf3-ad76-637b95496087 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:12.962863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 736/1479] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:36:12.968677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a8f9c-7581-4032-a275-80a4a4373f6e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:12.969124 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a8f9c-7581-4032-a275-80a4a4373f6e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:12.969229 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a8f9c-7581-4032-a275-80a4a4373f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:13.035013 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b85a8f9c-7581-4032-a275-80a4a4373f6e None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:36:13.036286 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b85a8f9c-7581-4032-a275-80a4a4373f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:13.132930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 744/1480] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:36:13.177293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19cea17-3b00-4bf3-ad76-637b95496087 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:13.178808 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b19cea17-3b00-4bf3-ad76-637b95496087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:13.188630 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:36:13.325864 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19cea17-3b00-4bf3-ad76-637b95496087 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:13.404913 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:36:13.438074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e082e9-642f-4bc1-a72f-34e73ef91286 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:13.438681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e082e9-642f-4bc1-a72f-34e73ef91286 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:13.438963 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e082e9-642f-4bc1-a72f-34e73ef91286 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:13.439268 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e082e9-642f-4bc1-a72f-34e73ef91286 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:13.440870 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-72e082e9-642f-4bc1-a72f-34e73ef91286 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:13.529185 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e082e9-642f-4bc1-a72f-34e73ef91286 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:13.530819 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e082e9-642f-4bc1-a72f-34e73ef91286 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:13.548393 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 737/1481] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:36:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:36:15.740781 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e76715-8629-4e31-9b4f-83f9fba02866 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:15.741345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e76715-8629-4e31-9b4f-83f9fba02866 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:15.741345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e76715-8629-4e31-9b4f-83f9fba02866 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:15.742663 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 745/1482] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:36:15.748616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82446e53-ccb3-43db-84c5-b23b5b544b0e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:15.755312 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82446e53-ccb3-43db-84c5-b23b5b544b0e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:15.755312 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82446e53-ccb3-43db-84c5-b23b5b544b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:15.816494 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-82446e53-ccb3-43db-84c5-b23b5b544b0e None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:36:15.817696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-82446e53-ccb3-43db-84c5-b23b5b544b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:15.864195 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:36:15.914426 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 738/1483] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:36:15.951317 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e76715-8629-4e31-9b4f-83f9fba02866 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:15.952930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-89e76715-8629-4e31-9b4f-83f9fba02866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:15.977166 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:36:16.055433 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e76715-8629-4e31-9b4f-83f9fba02866 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:16.133413 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e693d5c-7865-4753-a347-8b8caf3fff7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:16.134023 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e693d5c-7865-4753-a347-8b8caf3fff7f service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:16.134304 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e693d5c-7865-4753-a347-8b8caf3fff7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:16.134749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e693d5c-7865-4753-a347-8b8caf3fff7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:16.136376 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1e693d5c-7865-4753-a347-8b8caf3fff7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:16.211326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:36:16.247675 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e693d5c-7865-4753-a347-8b8caf3fff7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:16.249112 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e693d5c-7865-4753-a347-8b8caf3fff7f service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:16.267767 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 746/1484] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:36:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:36:18.360366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3002ca-929e-43da-b4a0-9e9060a2f737 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:18.360936 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3002ca-929e-43da-b4a0-9e9060a2f737 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:18.360936 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3002ca-929e-43da-b4a0-9e9060a2f737 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:18.362285 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 739/1485] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:36:18.367546 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab3a0f-30bb-4c73-934b-c3f18e2e722c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:18.367956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab3a0f-30bb-4c73-934b-c3f18e2e722c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:18.368098 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab3a0f-30bb-4c73-934b-c3f18e2e722c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:18.434451 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-eeab3a0f-30bb-4c73-934b-c3f18e2e722c None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:36:18.435698 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-eeab3a0f-30bb-4c73-934b-c3f18e2e722c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:18.530854 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 747/1486] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:36:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:36:18.569737 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3002ca-929e-43da-b4a0-9e9060a2f737 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:18.571364 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2b3002ca-929e-43da-b4a0-9e9060a2f737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:18.678335 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3002ca-929e-43da-b4a0-9e9060a2f737 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:18.750091 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:36:18.751926 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92142bad-bc97-4eed-9572-04ac0e83a33b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:18.752535 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92142bad-bc97-4eed-9572-04ac0e83a33b service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:18.752818 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92142bad-bc97-4eed-9572-04ac0e83a33b service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:18.753178 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92142bad-bc97-4eed-9572-04ac0e83a33b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:18.754721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-92142bad-bc97-4eed-9572-04ac0e83a33b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:18.864457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92142bad-bc97-4eed-9572-04ac0e83a33b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:18.865804 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92142bad-bc97-4eed-9572-04ac0e83a33b service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:18.884311 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 740/1487] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:36:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:36:21.047633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9436711-76ad-4f4c-a641-c88cd45555b8 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:21.048224 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9436711-76ad-4f4c-a641-c88cd45555b8 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:21.048224 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9436711-76ad-4f4c-a641-c88cd45555b8 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:21.049537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 748/1488] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:36:21.054676 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0191a710-f6dc-45b2-9113-3b8529c2000a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:21.055079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0191a710-f6dc-45b2-9113-3b8529c2000a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:21.055263 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0191a710-f6dc-45b2-9113-3b8529c2000a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:21.121908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0191a710-f6dc-45b2-9113-3b8529c2000a None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:36:21.123023 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0191a710-f6dc-45b2-9113-3b8529c2000a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:21.181312 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:36:21.225411 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 741/1489] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:36:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:36:21.266088 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9436711-76ad-4f4c-a641-c88cd45555b8 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:21.267703 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b9436711-76ad-4f4c-a641-c88cd45555b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:21.303504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:36:21.370922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9436711-76ad-4f4c-a641-c88cd45555b8 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:21.442603 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc96733f-a397-49b1-bf3d-465ac9d87d16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:21.443128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96733f-a397-49b1-bf3d-465ac9d87d16 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:21.443434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96733f-a397-49b1-bf3d-465ac9d87d16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:21.443701 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96733f-a397-49b1-bf3d-465ac9d87d16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:21.445284 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bc96733f-a397-49b1-bf3d-465ac9d87d16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:21.523756 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:36:21.553951 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc96733f-a397-49b1-bf3d-465ac9d87d16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:21.555461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc96733f-a397-49b1-bf3d-465ac9d87d16 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:21.575225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 749/1490] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:36:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:36:24.026547 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f455c-7b80-4427-94cb-f391c67d4000 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:24.027129 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f455c-7b80-4427-94cb-f391c67d4000 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:24.027129 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f455c-7b80-4427-94cb-f391c67d4000 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:24.028519 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 742/1491] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 12:36:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:36:24.033660 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b592a1-e0ac-4305-82d3-44b3b94af004 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:24.034099 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b592a1-e0ac-4305-82d3-44b3b94af004 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:24.034201 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b592a1-e0ac-4305-82d3-44b3b94af004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:24.100399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-31b592a1-e0ac-4305-82d3-44b3b94af004 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:36:24.101598 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-31b592a1-e0ac-4305-82d3-44b3b94af004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:24.200200 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 750/1492] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 12:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:36:24.240185 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f455c-7b80-4427-94cb-f391c67d4000 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:24.241752 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-da8f455c-7b80-4427-94cb-f391c67d4000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:24.345981 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f455c-7b80-4427-94cb-f391c67d4000 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:24.418688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ccd961-3ade-4a9d-852d-a764b5441fd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:24.419330 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccd961-3ade-4a9d-852d-a764b5441fd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:24.419609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccd961-3ade-4a9d-852d-a764b5441fd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:24.419880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccd961-3ade-4a9d-852d-a764b5441fd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:24.421476 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-85ccd961-3ade-4a9d-852d-a764b5441fd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:24.430410 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:36:24.529845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ccd961-3ade-4a9d-852d-a764b5441fd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:24.531364 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ccd961-3ade-4a9d-852d-a764b5441fd9 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:24.549486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 743/1493] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:36:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:36:34.198308 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=82 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:36:34.547154 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=50 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:53.139194 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c6829-d18e-46d3-8d4a-daf323333a63 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:53.139834 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c6829-d18e-46d3-8d4a-daf323333a63 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:53.139834 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c6829-d18e-46d3-8d4a-daf323333a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:53.215743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-000c6829-d18e-46d3-8d4a-daf323333a63 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:53.216953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-000c6829-d18e-46d3-8d4a-daf323333a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:53.312162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 751/1494] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:37:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:37:53.320516 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ce9aa-ea68-43a4-9015-d70888207abb None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:53.320998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ce9aa-ea68-43a4-9015-d70888207abb None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:53.321106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ce9aa-ea68-43a4-9015-d70888207abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:53.396231 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-679ce9aa-ea68-43a4-9015-d70888207abb None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:53.397372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-679ce9aa-ea68-43a4-9015-d70888207abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:53.494451 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 744/1495] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:37:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:37:53.499356 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c6829-d18e-46d3-8d4a-daf323333a63 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:53.500584 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-000c6829-d18e-46d3-8d4a-daf323333a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:53.512684 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c6829-d18e-46d3-8d4a-daf323333a63 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:53.574087 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:53.608235 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4eace9-3b23-483a-a556-1ccdacd7c3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:53.608873 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4eace9-3b23-483a-a556-1ccdacd7c3fa admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:53.609151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4eace9-3b23-483a-a556-1ccdacd7c3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:53.609421 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4eace9-3b23-483a-a556-1ccdacd7c3fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:53.610166 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4eace9-3b23-483a-a556-1ccdacd7c3fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:37:53.610471 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4eace9-3b23-483a-a556-1ccdacd7c3fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:53.611498 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: WARNING py.warnings [None req-6a4eace9-3b23-483a-a556-1ccdacd7c3fa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:37:53.611498 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: warnings.warn(msg) Aug 10 12:37:53.611498 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: Aug 10 12:37:53.612375 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4eace9-3b23-483a-a556-1ccdacd7c3fa admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:53.623721 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 752/1496] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:37:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:37:53.632235 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ce9aa-ea68-43a4-9015-d70888207abb None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:53.642032 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ce9aa-ea68-43a4-9015-d70888207abb None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:53.702925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:53.736575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1d263-8dbc-4f92-bbc9-f89a462e7e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:53.737173 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1d263-8dbc-4f92-bbc9-f89a462e7e27 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:53.737448 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1d263-8dbc-4f92-bbc9-f89a462e7e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:53.737714 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1d263-8dbc-4f92-bbc9-f89a462e7e27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:53.738331 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad1d263-8dbc-4f92-bbc9-f89a462e7e27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:53.740063 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: WARNING py.warnings [None req-bad1d263-8dbc-4f92-bbc9-f89a462e7e27 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:37:53.740063 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: warnings.warn(msg) Aug 10 12:37:53.740063 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: Aug 10 12:37:53.741237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad1d263-8dbc-4f92-bbc9-f89a462e7e27 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:53.890431 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 745/1497] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:37:53 2022] POST /identity/v3/projects => generated 377 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:37:53.894733 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4eace9-3b23-483a-a556-1ccdacd7c3fa admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:53.896315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6a4eace9-3b23-483a-a556-1ccdacd7c3fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:53.930462 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4eace9-3b23-483a-a556-1ccdacd7c3fa admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:54.027801 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d12c5f-abd8-49d0-bfe3-02e38e6f18a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:54.028414 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d12c5f-abd8-49d0-bfe3-02e38e6f18a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:54.028695 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d12c5f-abd8-49d0-bfe3-02e38e6f18a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:54.028970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d12c5f-abd8-49d0-bfe3-02e38e6f18a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:54.029622 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d12c5f-abd8-49d0-bfe3-02e38e6f18a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:54.030618 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: WARNING py.warnings [None req-76d12c5f-abd8-49d0-bfe3-02e38e6f18a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:37:54.030618 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: warnings.warn(msg) Aug 10 12:37:54.030618 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: Aug 10 12:37:54.031868 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d12c5f-abd8-49d0-bfe3-02e38e6f18a1 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:54.145169 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 753/1498] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:37:53 2022] POST /identity/v3/users => generated 289 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:37:54.149243 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1d263-8dbc-4f92-bbc9-f89a462e7e27 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:54.151518 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bad1d263-8dbc-4f92-bbc9-f89a462e7e27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:54.215063 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1d263-8dbc-4f92-bbc9-f89a462e7e27 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:54.300771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:54.311524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e30771-14c1-47f7-8295-ed2b6bea38e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:54.316052 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e30771-14c1-47f7-8295-ed2b6bea38e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:54.316052 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e30771-14c1-47f7-8295-ed2b6bea38e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:54.316052 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e30771-14c1-47f7-8295-ed2b6bea38e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:54.316052 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e30771-14c1-47f7-8295-ed2b6bea38e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:54.316052 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: WARNING py.warnings [None req-22e30771-14c1-47f7-8295-ed2b6bea38e6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:37:54.316052 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: warnings.warn(msg) Aug 10 12:37:54.316052 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: Aug 10 12:37:54.316773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e30771-14c1-47f7-8295-ed2b6bea38e6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:54.328370 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 746/1499] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:37:54.332469 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d12c5f-abd8-49d0-bfe3-02e38e6f18a1 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:54.333987 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-76d12c5f-abd8-49d0-bfe3-02e38e6f18a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:54.346186 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d12c5f-abd8-49d0-bfe3-02e38e6f18a1 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:54.371565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:54.440354 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e67831-b8c8-46bc-ad95-ecacb5b3c259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:54.440956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e67831-b8c8-46bc-ad95-ecacb5b3c259 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:54.441234 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e67831-b8c8-46bc-ad95-ecacb5b3c259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:54.441504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e67831-b8c8-46bc-ad95-ecacb5b3c259 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:54.442135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e67831-b8c8-46bc-ad95-ecacb5b3c259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:54.443138 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: WARNING py.warnings [None req-d3e67831-b8c8-46bc-ad95-ecacb5b3c259 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:37:54.443138 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: warnings.warn(msg) Aug 10 12:37:54.443138 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: Aug 10 12:37:54.444056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e67831-b8c8-46bc-ad95-ecacb5b3c259 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:54.457584 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 754/1500] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:37:54.461778 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e30771-14c1-47f7-8295-ed2b6bea38e6 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:54.463967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-22e30771-14c1-47f7-8295-ed2b6bea38e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:54.497437 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e30771-14c1-47f7-8295-ed2b6bea38e6 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:54.590526 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1c166-6442-4688-8777-25b12e6c2328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:54.591105 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1c166-6442-4688-8777-25b12e6c2328 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:54.591421 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1c166-6442-4688-8777-25b12e6c2328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:54.591699 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1c166-6442-4688-8777-25b12e6c2328 admin admin] PATH_INFO: `/v3/projects/299a7d0016654891bc1944163320260b/users/189cb475a6d4400ab967b7452a61a70b/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:54.613733 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1c166-6442-4688-8777-25b12e6c2328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=299a7d0016654891bc1944163320260b, user_id=189cb475a6d4400ab967b7452a61a70b, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:54.614767 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: WARNING py.warnings [None req-4be1c166-6442-4688-8777-25b12e6c2328 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:37:54.614767 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: warnings.warn(msg) Aug 10 12:37:54.614767 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: Aug 10 12:37:54.616785 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1c166-6442-4688-8777-25b12e6c2328 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:54.694693 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 747/1501] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:37:54 2022] PUT /identity/v3/projects/299a7d0016654891bc1944163320260b/users/189cb475a6d4400ab967b7452a61a70b/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:37:54.707504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166da1af-aa7e-4669-971b-fbe5f78e7a22 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:54.707796 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166da1af-aa7e-4669-971b-fbe5f78e7a22 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:54.708076 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166da1af-aa7e-4669-971b-fbe5f78e7a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:54.764687 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-166da1af-aa7e-4669-971b-fbe5f78e7a22 None None] MFA Rules not processed for user `189cb475a6d4400ab967b7452a61a70b`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:54.765951 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-166da1af-aa7e-4669-971b-fbe5f78e7a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:54.848492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 755/1502] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:37:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:37:54.861458 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d845b5-2e63-4e66-a209-8cfff6bcddbd None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:54.861748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d845b5-2e63-4e66-a209-8cfff6bcddbd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:54.862020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d845b5-2e63-4e66-a209-8cfff6bcddbd None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:54.863453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 748/1503] 10.222.0.49 () {54 vars in 964 bytes} [Wed Aug 10 12:37:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:37:54.868217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee0ba3-67b8-4b0d-b45c-098496135350 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:54.868507 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee0ba3-67b8-4b0d-b45c-098496135350 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:54.868788 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee0ba3-67b8-4b0d-b45c-098496135350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:54.899989 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:54.972571 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-17ee0ba3-67b8-4b0d-b45c-098496135350 None None] MFA Rules not processed for user `ab5a0d9c39d84cc7a91f5a752dcf2b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:54.973791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-17ee0ba3-67b8-4b0d-b45c-098496135350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:55.048046 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 756/1504] 10.222.0.49 () {58 vars in 1061 bytes} [Wed Aug 10 12:37:54 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:37:55.055887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d845b5-2e63-4e66-a209-8cfff6bcddbd None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:55.057588 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-58d845b5-2e63-4e66-a209-8cfff6bcddbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:55.107203 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:55.161394 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d845b5-2e63-4e66-a209-8cfff6bcddbd None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:55.236119 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0216262-e404-4410-990b-dfb735d63883 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:55.236722 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0216262-e404-4410-990b-dfb735d63883 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:55.236996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0216262-e404-4410-990b-dfb735d63883 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:55.237372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0216262-e404-4410-990b-dfb735d63883 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:55.238962 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e0216262-e404-4410-990b-dfb735d63883 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:55.336603 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0216262-e404-4410-990b-dfb735d63883 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:55.338144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0216262-e404-4410-990b-dfb735d63883 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:55.341587 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:55.355733 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 749/1505] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:37:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:37:55.402299 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a236664-a079-49ed-920e-957d855020b5 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:55.402738 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a236664-a079-49ed-920e-957d855020b5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:55.402849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a236664-a079-49ed-920e-957d855020b5 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:55.404131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 757/1506] 10.222.0.49 () {52 vars in 830 bytes} [Wed Aug 10 12:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:37:55.407647 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0216262-e404-4410-990b-dfb735d63883 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:55.408911 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e0216262-e404-4410-990b-dfb735d63883 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:55.507035 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0216262-e404-4410-990b-dfb735d63883 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:55.599150 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a940d-1ed1-499b-99d9-5d2e6668d7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:55.599805 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a940d-1ed1-499b-99d9-5d2e6668d7ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:55.600086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a940d-1ed1-499b-99d9-5d2e6668d7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:55.600351 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a940d-1ed1-499b-99d9-5d2e6668d7ed admin admin] PATH_INFO: `/v3/users/189cb475a6d4400ab967b7452a61a70b` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:55.612713 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a940d-1ed1-499b-99d9-5d2e6668d7ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=189cb475a6d4400ab967b7452a61a70b)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:55.613698 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: WARNING py.warnings [None req-4f6a940d-1ed1-499b-99d9-5d2e6668d7ed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:37:55.613698 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: warnings.warn(msg) Aug 10 12:37:55.613698 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: Aug 10 12:37:55.614900 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a940d-1ed1-499b-99d9-5d2e6668d7ed admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:55.625449 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:55.831118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4f6a940d-1ed1-499b-99d9-5d2e6668d7ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '189cb475a6d4400ab967b7452a61a70b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:55.903878 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4f6a940d-1ed1-499b-99d9-5d2e6668d7ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '189cb475a6d4400ab967b7452a61a70b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:55.904414 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4f6a940d-1ed1-499b-99d9-5d2e6668d7ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '189cb475a6d4400ab967b7452a61a70b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:55.904842 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4f6a940d-1ed1-499b-99d9-5d2e6668d7ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '189cb475a6d4400ab967b7452a61a70b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:55.920441 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4f6a940d-1ed1-499b-99d9-5d2e6668d7ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '189cb475a6d4400ab967b7452a61a70b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:55.949197 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 750/1507] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:37:55 2022] DELETE /identity/v3/users/189cb475a6d4400ab967b7452a61a70b => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:37:55.953696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6deb5-be71-454b-93ac-6dbb84d2eff0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:55.953986 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6deb5-be71-454b-93ac-6dbb84d2eff0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:55.954262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6deb5-be71-454b-93ac-6dbb84d2eff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:56.010965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3bd6deb5-be71-454b-93ac-6dbb84d2eff0 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:56.012204 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3bd6deb5-be71-454b-93ac-6dbb84d2eff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:56.085915 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:56.098985 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 758/1508] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:37:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:37:56.110175 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a940d-1ed1-499b-99d9-5d2e6668d7ed admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:56.111432 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4f6a940d-1ed1-499b-99d9-5d2e6668d7ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:56.209345 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a940d-1ed1-499b-99d9-5d2e6668d7ed admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:56.263791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:56.283224 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe54983-1b6f-4efc-aedc-0f78f12d6ec0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:56.283843 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe54983-1b6f-4efc-aedc-0f78f12d6ec0 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:56.284123 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe54983-1b6f-4efc-aedc-0f78f12d6ec0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:56.284395 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe54983-1b6f-4efc-aedc-0f78f12d6ec0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:56.285988 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6fe54983-1b6f-4efc-aedc-0f78f12d6ec0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:56.394877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe54983-1b6f-4efc-aedc-0f78f12d6ec0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:56.396346 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe54983-1b6f-4efc-aedc-0f78f12d6ec0 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:56.415247 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 751/1509] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:37:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:37:56.900122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6deb5-be71-454b-93ac-6dbb84d2eff0 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:56.905007 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3bd6deb5-be71-454b-93ac-6dbb84d2eff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:56.913115 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6deb5-be71-454b-93ac-6dbb84d2eff0 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:57.010843 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f0544-2590-4581-b3a3-54633a0662d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:57.011506 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f0544-2590-4581-b3a3-54633a0662d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:57.011786 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f0544-2590-4581-b3a3-54633a0662d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:57.012057 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f0544-2590-4581-b3a3-54633a0662d6 admin admin] PATH_INFO: `/v3/projects/299a7d0016654891bc1944163320260b` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:57.021747 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7f0544-2590-4581-b3a3-54633a0662d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=299a7d0016654891bc1944163320260b)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:57.022769 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: WARNING py.warnings [None req-bd7f0544-2590-4581-b3a3-54633a0662d6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:37:57.022769 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: warnings.warn(msg) Aug 10 12:37:57.022769 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: Aug 10 12:37:57.024000 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7f0544-2590-4581-b3a3-54633a0662d6 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:57.119493 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bd7f0544-2590-4581-b3a3-54633a0662d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '299a7d0016654891bc1944163320260b', 'request_id': 'req-bd7f0544-2590-4581-b3a3-54633a0662d6'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:57.125513 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bd7f0544-2590-4581-b3a3-54633a0662d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '299a7d0016654891bc1944163320260b', 'request_id': 'req-bd7f0544-2590-4581-b3a3-54633a0662d6'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:57.172484 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bd7f0544-2590-4581-b3a3-54633a0662d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '299a7d0016654891bc1944163320260b', 'request_id': 'req-bd7f0544-2590-4581-b3a3-54633a0662d6'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:57.202782 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bd7f0544-2590-4581-b3a3-54633a0662d6 admin admin] The token cache is being invalidate because project 299a7d0016654891bc1944163320260b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:37:57.203078 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bd7f0544-2590-4581-b3a3-54633a0662d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:57.203633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bd7f0544-2590-4581-b3a3-54633a0662d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:57.204947 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 759/1510] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:37:56 2022] DELETE /identity/v3/projects/299a7d0016654891bc1944163320260b => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:06.412929 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=62 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:07.201808 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:26.063680 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16699b-09ac-4047-a0d9-a2df41171848 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:26.064312 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16699b-09ac-4047-a0d9-a2df41171848 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:26.064312 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16699b-09ac-4047-a0d9-a2df41171848 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:26.065565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 12:39:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:39:26.070425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4df91-1b28-46c4-af21-547329e2d180 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:26.070851 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4df91-1b28-46c4-af21-547329e2d180 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:26.070976 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4df91-1b28-46c4-af21-547329e2d180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:26.153477 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-55e4df91-1b28-46c4-af21-547329e2d180 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:26.154593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-55e4df91-1b28-46c4-af21-547329e2d180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:26.258902 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 12:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:39:26.414902 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16699b-09ac-4047-a0d9-a2df41171848 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:26.417276 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1b16699b-09ac-4047-a0d9-a2df41171848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:26.539186 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16699b-09ac-4047-a0d9-a2df41171848 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:26.544960 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa565233-78b6-4501-be7e-964d119243ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:26.557447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa565233-78b6-4501-be7e-964d119243ea service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:26.557447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa565233-78b6-4501-be7e-964d119243ea service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:26.557447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa565233-78b6-4501-be7e-964d119243ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:26.557447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa565233-78b6-4501-be7e-964d119243ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:26.557447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa565233-78b6-4501-be7e-964d119243ea service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:26.565800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 753/1513] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:39:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:39:27.401239 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d386b8-2de0-415d-9f10-d50133bea19b None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:27.401770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d386b8-2de0-415d-9f10-d50133bea19b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:27.401770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d386b8-2de0-415d-9f10-d50133bea19b None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:27.403045 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 761/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 12:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:39:27.408694 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78cf98-3e44-45cc-9bc8-ca149c611d8a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:27.409146 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78cf98-3e44-45cc-9bc8-ca149c611d8a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:27.409254 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78cf98-3e44-45cc-9bc8-ca149c611d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:27.428766 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5b78cf98-3e44-45cc-9bc8-ca149c611d8a None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:27.430100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5b78cf98-3e44-45cc-9bc8-ca149c611d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:27.439109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 754/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 12:39:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:39:27.592375 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d386b8-2de0-415d-9f10-d50133bea19b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:27.602898 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7f166-e981-4a7b-92fe-6b4baae9a8d0 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:27.603195 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7f166-e981-4a7b-92fe-6b4baae9a8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:27.603491 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7f166-e981-4a7b-92fe-6b4baae9a8d0 None None] PATH_INFO: `/v3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:27.604813 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 755/1516] 10.222.0.55 () {54 vars in 925 bytes} [Wed Aug 10 12:39:27 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:39:27.607351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d386b8-2de0-415d-9f10-d50133bea19b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:27.611007 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2e84c-685b-4976-888c-3d24ce219efd None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:27.611322 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2e84c-685b-4976-888c-3d24ce219efd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:27.611595 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2e84c-685b-4976-888c-3d24ce219efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:27.613376 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6c2c2-f762-4ed8-9856-2a421c0e56c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:27.614013 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6c2c2-f762-4ed8-9856-2a421c0e56c1 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:27.614296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6c2c2-f762-4ed8-9856-2a421c0e56c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:27.614572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6c2c2-f762-4ed8-9856-2a421c0e56c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:27.623520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6c2c2-f762-4ed8-9856-2a421c0e56c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:27.626191 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6c2c2-f762-4ed8-9856-2a421c0e56c1 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:27.636618 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 762/1517] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:39:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:39:27.673608 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-cee2e84c-685b-4976-888c-3d24ce219efd None None] MFA Rules not processed for user `3ff152e22374407eb72caf648a763107`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:27.674868 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cee2e84c-685b-4976-888c-3d24ce219efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:27.734068 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:27.754841 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 756/1518] 10.222.0.55 () {58 vars in 1015 bytes} [Wed Aug 10 12:39:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:39:31.720056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bbcdc0-bc3e-4fbb-b382-ab68b154f608 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:31.720566 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bbcdc0-bc3e-4fbb-b382-ab68b154f608 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:31.720566 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bbcdc0-bc3e-4fbb-b382-ab68b154f608 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:31.721847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 763/1519] 10.222.0.49 () {54 vars in 961 bytes} [Wed Aug 10 12:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:39:31.726714 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a20e6-f6b6-4752-b0e9-bc6bf4c43c06 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:31.727108 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a20e6-f6b6-4752-b0e9-bc6bf4c43c06 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:31.727293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a20e6-f6b6-4752-b0e9-bc6bf4c43c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:31.746680 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-af6a20e6-f6b6-4752-b0e9-bc6bf4c43c06 None None] MFA Rules not processed for user `3ff152e22374407eb72caf648a763107`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:31.747922 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-af6a20e6-f6b6-4752-b0e9-bc6bf4c43c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:31.756953 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 757/1520] 10.222.0.49 () {58 vars in 1058 bytes} [Wed Aug 10 12:39:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:39:31.764196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bbcdc0-bc3e-4fbb-b382-ab68b154f608 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:31.778479 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bbcdc0-bc3e-4fbb-b382-ab68b154f608 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:31.783440 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfe08d-da72-4965-8b81-7a8a6e49d018 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff152e22374407eb72caf648a763107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:31.784054 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfe08d-da72-4965-8b81-7a8a6e49d018 service placement] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:31.784334 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfe08d-da72-4965-8b81-7a8a6e49d018 service placement] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:31.784605 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfe08d-da72-4965-8b81-7a8a6e49d018 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:31.792759 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcfe08d-da72-4965-8b81-7a8a6e49d018 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:31.794325 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcfe08d-da72-4965-8b81-7a8a6e49d018 service placement] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:31.804085 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 764/1521] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:39:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:39:38.874953 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64c90b-9f3d-47fd-a92e-fbf32b828cac None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:38.875549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64c90b-9f3d-47fd-a92e-fbf32b828cac None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:38.875549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64c90b-9f3d-47fd-a92e-fbf32b828cac None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:38.876868 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 758/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 12:39:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:39:38.882660 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a119e-dfd9-496f-9c94-2d0e5e871ada None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:38.891321 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a119e-dfd9-496f-9c94-2d0e5e871ada None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:38.891321 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a119e-dfd9-496f-9c94-2d0e5e871ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:38.903317 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-585a119e-dfd9-496f-9c94-2d0e5e871ada None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:38.904559 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-585a119e-dfd9-496f-9c94-2d0e5e871ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:38.914039 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 765/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 12:39:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:39:39.072359 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64c90b-9f3d-47fd-a92e-fbf32b828cac None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:39.086706 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64c90b-9f3d-47fd-a92e-fbf32b828cac None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:39.092267 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665333e-d5b1-4f12-834a-eb76e2ac5533 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:39.092860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665333e-d5b1-4f12-834a-eb76e2ac5533 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:39.093146 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665333e-d5b1-4f12-834a-eb76e2ac5533 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:39.093440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665333e-d5b1-4f12-834a-eb76e2ac5533 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:39.103085 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6665333e-d5b1-4f12-834a-eb76e2ac5533 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:39.104833 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6665333e-d5b1-4f12-834a-eb76e2ac5533 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:39.115504 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 759/1524] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:39:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:39:39.933739 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8d411-a3c7-4900-8441-fa5cc5495521 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:39.934321 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8d411-a3c7-4900-8441-fa5cc5495521 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:39.934321 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8d411-a3c7-4900-8441-fa5cc5495521 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:39.935701 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 766/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 12:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:39:39.940494 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208e8d7-d988-4440-8d15-c4ea5cd29ed1 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:39.941434 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208e8d7-d988-4440-8d15-c4ea5cd29ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:39.941697 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208e8d7-d988-4440-8d15-c4ea5cd29ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:39.961720 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4208e8d7-d988-4440-8d15-c4ea5cd29ed1 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:39.962910 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4208e8d7-d988-4440-8d15-c4ea5cd29ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:39.970560 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 760/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 12:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:39:40.121797 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8d411-a3c7-4900-8441-fa5cc5495521 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:40.136678 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8d411-a3c7-4900-8441-fa5cc5495521 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:40.142098 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ac3b7-112e-4de7-bf28-847266faf02a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:40.142687 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ac3b7-112e-4de7-bf28-847266faf02a service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:40.142966 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ac3b7-112e-4de7-bf28-847266faf02a service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:40.143271 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ac3b7-112e-4de7-bf28-847266faf02a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:40.151537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ac3b7-112e-4de7-bf28-847266faf02a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:40.153209 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ac3b7-112e-4de7-bf28-847266faf02a service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:40.162924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 767/1527] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:39:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:39:43.999980 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bc396-476e-4b68-b71e-065ba401d84f None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:44.000550 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bc396-476e-4b68-b71e-065ba401d84f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:44.000550 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bc396-476e-4b68-b71e-065ba401d84f None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:44.001854 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:39:44.007670 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f61670-c0bc-4ca3-8f9c-d7af0c392125 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:44.008108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f61670-c0bc-4ca3-8f9c-d7af0c392125 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:44.008205 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f61670-c0bc-4ca3-8f9c-d7af0c392125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:44.027853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-11f61670-c0bc-4ca3-8f9c-d7af0c392125 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:44.033199 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-11f61670-c0bc-4ca3-8f9c-d7af0c392125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:44.037287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:39:44.110806 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88bc396-476e-4b68-b71e-065ba401d84f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:44.113089 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f88bc396-476e-4b68-b71e-065ba401d84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:44.190287 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88bc396-476e-4b68-b71e-065ba401d84f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:44.195026 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a3004-5737-41cb-a867-535f207f8042 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:44.195660 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a3004-5737-41cb-a867-535f207f8042 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:44.195943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a3004-5737-41cb-a867-535f207f8042 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:44.196213 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a3004-5737-41cb-a867-535f207f8042 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:44.204962 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99a3004-5737-41cb-a867-535f207f8042 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:44.206977 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99a3004-5737-41cb-a867-535f207f8042 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:44.216154 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 762/1530] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:39:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:39:46.234987 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735cabc-5c5e-4ee2-abeb-9d48c2c63c45 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:46.235603 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735cabc-5c5e-4ee2-abeb-9d48c2c63c45 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:46.235603 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735cabc-5c5e-4ee2-abeb-9d48c2c63c45 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:46.236918 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:39:46.242270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ec502-73d1-45bb-b831-a8e69a146a85 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:46.242718 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ec502-73d1-45bb-b831-a8e69a146a85 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:46.242836 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ec502-73d1-45bb-b831-a8e69a146a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:46.262121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-364ec502-73d1-45bb-b831-a8e69a146a85 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:46.263364 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-364ec502-73d1-45bb-b831-a8e69a146a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:46.271782 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:39:46.307949 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735cabc-5c5e-4ee2-abeb-9d48c2c63c45 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:46.319472 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735cabc-5c5e-4ee2-abeb-9d48c2c63c45 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:46.322978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8585f7-d155-48b3-89b8-dd3f60c8c16b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:46.323599 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8585f7-d155-48b3-89b8-dd3f60c8c16b service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:46.323879 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8585f7-d155-48b3-89b8-dd3f60c8c16b service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:46.324154 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8585f7-d155-48b3-89b8-dd3f60c8c16b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:46.332964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8585f7-d155-48b3-89b8-dd3f60c8c16b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:46.334377 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8585f7-d155-48b3-89b8-dd3f60c8c16b service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:46.343599 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 770/1533] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:39:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:39:50.241020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29722f-a8a6-4aea-955f-16b151c2f9b5 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:50.241580 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29722f-a8a6-4aea-955f-16b151c2f9b5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:50.241580 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29722f-a8a6-4aea-955f-16b151c2f9b5 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:50.242817 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:39:50.248062 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4f91a-7c91-4e8e-bfed-6b015c42fe40 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:50.248536 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4f91a-7c91-4e8e-bfed-6b015c42fe40 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:50.248643 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4f91a-7c91-4e8e-bfed-6b015c42fe40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:50.257041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:50.270054 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-05c4f91a-7c91-4e8e-bfed-6b015c42fe40 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:50.271604 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-05c4f91a-7c91-4e8e-bfed-6b015c42fe40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:50.279660 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:39:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:39:50.585058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29722f-a8a6-4aea-955f-16b151c2f9b5 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:50.586782 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1a29722f-a8a6-4aea-955f-16b151c2f9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:50.664834 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29722f-a8a6-4aea-955f-16b151c2f9b5 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:50.668451 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd759ad-e7c7-4e17-b632-b94d8cb0af15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:50.669121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd759ad-e7c7-4e17-b632-b94d8cb0af15 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:50.669405 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd759ad-e7c7-4e17-b632-b94d8cb0af15 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:50.669676 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd759ad-e7c7-4e17-b632-b94d8cb0af15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:50.679509 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd759ad-e7c7-4e17-b632-b94d8cb0af15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:50.681047 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd759ad-e7c7-4e17-b632-b94d8cb0af15 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:50.690138 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 765/1536] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:39:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:39:52.712112 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9894a-0a30-44f0-b7d7-635f5eb7c0b6 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:52.712649 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9894a-0a30-44f0-b7d7-635f5eb7c0b6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:52.712649 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9894a-0a30-44f0-b7d7-635f5eb7c0b6 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:52.713936 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:39:52.719006 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44850184-e0bd-49ea-9894-d613d360b217 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:52.719431 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44850184-e0bd-49ea-9894-d613d360b217 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:52.719592 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44850184-e0bd-49ea-9894-d613d360b217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:52.740209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-44850184-e0bd-49ea-9894-d613d360b217 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:52.741415 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-44850184-e0bd-49ea-9894-d613d360b217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:52.749228 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:39:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:39:53.101683 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9894a-0a30-44f0-b7d7-635f5eb7c0b6 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:53.103422 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-16c9894a-0a30-44f0-b7d7-635f5eb7c0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:53.118064 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9894a-0a30-44f0-b7d7-635f5eb7c0b6 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:53.121688 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51ed03-c06c-4f3f-99db-55fbdbcd094b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:53.122318 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51ed03-c06c-4f3f-99db-55fbdbcd094b service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:53.122595 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51ed03-c06c-4f3f-99db-55fbdbcd094b service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:53.122866 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51ed03-c06c-4f3f-99db-55fbdbcd094b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:53.133383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51ed03-c06c-4f3f-99db-55fbdbcd094b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:53.134924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51ed03-c06c-4f3f-99db-55fbdbcd094b service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:53.145148 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 773/1539] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:39:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:40:02.735437 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:40:03.140520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=8 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:17.803119 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3213c04-ef83-4c4f-b435-209f70e96af4 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:17.804857 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3213c04-ef83-4c4f-b435-209f70e96af4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:17.804857 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3213c04-ef83-4c4f-b435-209f70e96af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:17.831114 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e3213c04-ef83-4c4f-b435-209f70e96af4 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:17.834456 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e3213c04-ef83-4c4f-b435-209f70e96af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:17.844107 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 767/1540] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:17.854351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff00b37-abc6-4e23-8ad1-9c72d0d83ebf None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:17.854968 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff00b37-abc6-4e23-8ad1-9c72d0d83ebf None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:17.854968 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff00b37-abc6-4e23-8ad1-9c72d0d83ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:17.882866 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-2ff00b37-abc6-4e23-8ad1-9c72d0d83ebf None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:17.884153 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2ff00b37-abc6-4e23-8ad1-9c72d0d83ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:17.897667 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 774/1541] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:17.903564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3213c04-ef83-4c4f-b435-209f70e96af4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:17.917024 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3213c04-ef83-4c4f-b435-209f70e96af4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:17.924004 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cdf2ea-7944-43b2-bcde-98a9e2551ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:17.926163 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdf2ea-7944-43b2-bcde-98a9e2551ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:17.926163 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdf2ea-7944-43b2-bcde-98a9e2551ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:17.926163 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdf2ea-7944-43b2-bcde-98a9e2551ef2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:17.926163 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cdf2ea-7944-43b2-bcde-98a9e2551ef2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:41:17.926579 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cdf2ea-7944-43b2-bcde-98a9e2551ef2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:17.927264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: WARNING py.warnings [None req-f5cdf2ea-7944-43b2-bcde-98a9e2551ef2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:41:17.927264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: warnings.warn(msg) Aug 10 12:41:17.927264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: Aug 10 12:41:17.928427 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cdf2ea-7944-43b2-bcde-98a9e2551ef2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:17.948609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 768/1542] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:41:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:41:17.957661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff00b37-abc6-4e23-8ad1-9c72d0d83ebf None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:17.976271 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff00b37-abc6-4e23-8ad1-9c72d0d83ebf None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:17.983474 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3973f421-0dde-4a33-b8fa-e033e0ba2be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:17.984101 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973f421-0dde-4a33-b8fa-e033e0ba2be9 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:17.984381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973f421-0dde-4a33-b8fa-e033e0ba2be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:17.984654 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973f421-0dde-4a33-b8fa-e033e0ba2be9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:17.985676 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3973f421-0dde-4a33-b8fa-e033e0ba2be9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:17.986714 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: WARNING py.warnings [None req-3973f421-0dde-4a33-b8fa-e033e0ba2be9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:41:17.986714 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: warnings.warn(msg) Aug 10 12:41:17.986714 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: Aug 10 12:41:17.987937 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3973f421-0dde-4a33-b8fa-e033e0ba2be9 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:18.116918 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 775/1543] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:41:17 2022] POST /identity/v3/projects => generated 383 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:18.121502 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cdf2ea-7944-43b2-bcde-98a9e2551ef2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:18.135194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cdf2ea-7944-43b2-bcde-98a9e2551ef2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:18.218389 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fec8e3-e95c-4de2-9f04-e2477570d02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:18.219370 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fec8e3-e95c-4de2-9f04-e2477570d02c admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:18.219659 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fec8e3-e95c-4de2-9f04-e2477570d02c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:18.219939 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fec8e3-e95c-4de2-9f04-e2477570d02c admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:18.220586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fec8e3-e95c-4de2-9f04-e2477570d02c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:18.221629 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: WARNING py.warnings [None req-c7fec8e3-e95c-4de2-9f04-e2477570d02c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:41:18.221629 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: warnings.warn(msg) Aug 10 12:41:18.221629 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: Aug 10 12:41:18.223423 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fec8e3-e95c-4de2-9f04-e2477570d02c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:18.289146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8bee8-85c6-4726-8e28-452f0ec29daf None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:18.289487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8bee8-85c6-4726-8e28-452f0ec29daf None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:18.289704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8bee8-85c6-4726-8e28-452f0ec29daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:18.326379 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f5e8bee8-85c6-4726-8e28-452f0ec29daf None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:18.329447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f5e8bee8-85c6-4726-8e28-452f0ec29daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:18.337503 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 769/1544] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:41:18 2022] POST /identity/v3/users => generated 292 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:18.338791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15937f19-15b7-46f0-97cd-3df697042593 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:18.339074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15937f19-15b7-46f0-97cd-3df697042593 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:18.339374 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15937f19-15b7-46f0-97cd-3df697042593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:18.349459 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 776/1545] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 10 12:41:18.353964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8bee8-85c6-4726-8e28-452f0ec29daf None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:18.370007 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8bee8-85c6-4726-8e28-452f0ec29daf None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:18.381479 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-15937f19-15b7-46f0-97cd-3df697042593 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:18.382703 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-15937f19-15b7-46f0-97cd-3df697042593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:18.386425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b09b0-085f-48ca-ae7c-90dca21bb4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:18.387111 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b09b0-085f-48ca-ae7c-90dca21bb4cd admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:18.387451 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b09b0-085f-48ca-ae7c-90dca21bb4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:18.387730 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b09b0-085f-48ca-ae7c-90dca21bb4cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:18.388387 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b09b0-085f-48ca-ae7c-90dca21bb4cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:18.389993 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b09b0-085f-48ca-ae7c-90dca21bb4cd admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:18.392141 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 770/1546] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:18.393506 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da90a6-c249-4bc6-8308-3dd64744410a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:18.393953 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da90a6-c249-4bc6-8308-3dd64744410a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:18.395042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da90a6-c249-4bc6-8308-3dd64744410a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:18.409227 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 777/1547] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:18.411080 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b74af-ef74-4056-a0f1-77742845da49 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:18.415689 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b74af-ef74-4056-a0f1-77742845da49 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:18.416066 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b74af-ef74-4056-a0f1-77742845da49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:18.425363 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a0da90a6-c249-4bc6-8308-3dd64744410a None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:18.427214 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a0da90a6-c249-4bc6-8308-3dd64744410a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:18.435761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 771/1548] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:18.437362 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5379fc3-f70f-4f5f-8341-7e0e29dfc160 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:18.437817 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5379fc3-f70f-4f5f-8341-7e0e29dfc160 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:18.438256 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5379fc3-f70f-4f5f-8341-7e0e29dfc160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:18.459860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a5379fc3-f70f-4f5f-8341-7e0e29dfc160 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:18.461547 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a5379fc3-f70f-4f5f-8341-7e0e29dfc160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:18.463438 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6c0b74af-ef74-4056-a0f1-77742845da49 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:18.464640 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6c0b74af-ef74-4056-a0f1-77742845da49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:18.471576 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 772/1549] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:18.477143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 778/1550] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:18.477143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b74af-ef74-4056-a0f1-77742845da49 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:18.477705 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8798f7-f47a-42ab-9bec-98c119069f20 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:18.478291 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8798f7-f47a-42ab-9bec-98c119069f20 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:18.478766 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8798f7-f47a-42ab-9bec-98c119069f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:18.485471 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b74af-ef74-4056-a0f1-77742845da49 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:18.490654 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da5427-e047-46d9-89e8-91be3e2713fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:18.491379 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da5427-e047-46d9-89e8-91be3e2713fc admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:18.491730 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da5427-e047-46d9-89e8-91be3e2713fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:18.492078 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da5427-e047-46d9-89e8-91be3e2713fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:18.492788 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da5427-e047-46d9-89e8-91be3e2713fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:18.494487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da5427-e047-46d9-89e8-91be3e2713fc admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:18.501590 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6e8798f7-f47a-42ab-9bec-98c119069f20 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:18.503049 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6e8798f7-f47a-42ab-9bec-98c119069f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:18.506396 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 779/1551] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:18.508192 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ea84d-b50b-4939-938e-e6b782716f72 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:18.508550 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ea84d-b50b-4939-938e-e6b782716f72 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:18.508903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ea84d-b50b-4939-938e-e6b782716f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:18.513468 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 773/1552] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:18.518378 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456fb30-7d0b-4def-8a62-e7449cf1ef6a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:18.519067 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456fb30-7d0b-4def-8a62-e7449cf1ef6a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:18.520086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456fb30-7d0b-4def-8a62-e7449cf1ef6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:18.535369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6d9ea84d-b50b-4939-938e-e6b782716f72 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:18.536511 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6d9ea84d-b50b-4939-938e-e6b782716f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:18.544998 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 780/1553] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:18.545674 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ea84d-b50b-4939-938e-e6b782716f72 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:18.550605 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b456fb30-7d0b-4def-8a62-e7449cf1ef6a None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:18.553024 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b456fb30-7d0b-4def-8a62-e7449cf1ef6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:18.555717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ea84d-b50b-4939-938e-e6b782716f72 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:18.568024 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc00fca7-6c33-4fcc-8153-06e86008bec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:18.568513 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00fca7-6c33-4fcc-8153-06e86008bec8 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:18.568802 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00fca7-6c33-4fcc-8153-06e86008bec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:18.569087 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00fca7-6c33-4fcc-8153-06e86008bec8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:18.569728 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc00fca7-6c33-4fcc-8153-06e86008bec8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:41:18.570004 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 774/1554] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:18.570119 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc00fca7-6c33-4fcc-8153-06e86008bec8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:18.571346 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726efe4-4697-4047-be88-a7d1ad162983 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:18.571639 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726efe4-4697-4047-be88-a7d1ad162983 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:18.571814 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc00fca7-6c33-4fcc-8153-06e86008bec8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:18.571924 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726efe4-4697-4047-be88-a7d1ad162983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:18.587037 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 781/1555] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:41:18.587037 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc00fca7-6c33-4fcc-8153-06e86008bec8 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:18.592056 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f726efe4-4697-4047-be88-a7d1ad162983 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:18.593261 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f726efe4-4697-4047-be88-a7d1ad162983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:18.596264 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc00fca7-6c33-4fcc-8153-06e86008bec8 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:18.600788 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84113e7-0b32-45ac-b393-241dc48088a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:18.601460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 775/1556] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:18.601595 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84113e7-0b32-45ac-b393-241dc48088a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:18.601698 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84113e7-0b32-45ac-b393-241dc48088a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:18.601956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84113e7-0b32-45ac-b393-241dc48088a5 admin admin] PATH_INFO: `/v3/projects/578ea63d03fc48c881899b6482d954e0/users/5ce0e999ef0b4b26ac24d703661d51a5/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:18.602223 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726efe4-4697-4047-be88-a7d1ad162983 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:18.611848 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726efe4-4697-4047-be88-a7d1ad162983 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:18.617262 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9286ed0-e7ed-4773-b1e3-65d2f38323da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:18.617437 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84113e7-0b32-45ac-b393-241dc48088a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=578ea63d03fc48c881899b6482d954e0, user_id=5ce0e999ef0b4b26ac24d703661d51a5, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:18.617860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9286ed0-e7ed-4773-b1e3-65d2f38323da admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:18.618140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9286ed0-e7ed-4773-b1e3-65d2f38323da admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:18.618413 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9286ed0-e7ed-4773-b1e3-65d2f38323da admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:18.618531 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: WARNING py.warnings [None req-c84113e7-0b32-45ac-b393-241dc48088a5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:41:18.618531 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: warnings.warn(msg) Aug 10 12:41:18.618531 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: Aug 10 12:41:18.619050 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9286ed0-e7ed-4773-b1e3-65d2f38323da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:41:18.619380 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9286ed0-e7ed-4773-b1e3-65d2f38323da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:18.620451 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84113e7-0b32-45ac-b393-241dc48088a5 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:18.621010 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9286ed0-e7ed-4773-b1e3-65d2f38323da admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:18.632936 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 776/1557] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:41:18.634117 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418af6f-de8b-4284-a4c9-ec4d465b2e45 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:18.636407 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418af6f-de8b-4284-a4c9-ec4d465b2e45 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:18.636407 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418af6f-de8b-4284-a4c9-ec4d465b2e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:18.653981 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f418af6f-de8b-4284-a4c9-ec4d465b2e45 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:18.655198 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f418af6f-de8b-4284-a4c9-ec4d465b2e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:18.663677 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 777/1558] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:18.668745 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418af6f-de8b-4284-a4c9-ec4d465b2e45 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:18.673868 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418af6f-de8b-4284-a4c9-ec4d465b2e45 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:18.678791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d40a69-1558-40ff-afa2-bcb4afa87e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:18.679407 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d40a69-1558-40ff-afa2-bcb4afa87e7f admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:18.679687 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d40a69-1558-40ff-afa2-bcb4afa87e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:18.679964 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d40a69-1558-40ff-afa2-bcb4afa87e7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:18.680619 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d40a69-1558-40ff-afa2-bcb4afa87e7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:41:18.680920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d40a69-1558-40ff-afa2-bcb4afa87e7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:18.682529 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d40a69-1558-40ff-afa2-bcb4afa87e7f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:18.694521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 778/1559] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:41:18.699549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515682c-33d9-461a-948c-81d3a37f7073 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:18.699549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515682c-33d9-461a-948c-81d3a37f7073 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:18.699549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515682c-33d9-461a-948c-81d3a37f7073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:18.715057 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a515682c-33d9-461a-948c-81d3a37f7073 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:18.716279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a515682c-33d9-461a-948c-81d3a37f7073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:18.721707 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 782/1560] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:41:18 2022] PUT /identity/v3/projects/578ea63d03fc48c881899b6482d954e0/users/5ce0e999ef0b4b26ac24d703661d51a5/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:41:18.725937 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52521d76-1eb8-4ede-a687-431b0f192bd4 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:18.725937 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52521d76-1eb8-4ede-a687-431b0f192bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:18.725937 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52521d76-1eb8-4ede-a687-431b0f192bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:18.757457 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-52521d76-1eb8-4ede-a687-431b0f192bd4 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:18.757457 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-52521d76-1eb8-4ede-a687-431b0f192bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:18.770247 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:18.779684 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 779/1561] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:18.780615 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a515682c-33d9-461a-948c-81d3a37f7073 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:18.791921 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a515682c-33d9-461a-948c-81d3a37f7073 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:18.797226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddfb42e-34e5-4a34-b501-63dc8850263f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:18.797874 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfb42e-34e5-4a34-b501-63dc8850263f admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:18.798231 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfb42e-34e5-4a34-b501-63dc8850263f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:18.798570 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfb42e-34e5-4a34-b501-63dc8850263f admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:18.799335 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddfb42e-34e5-4a34-b501-63dc8850263f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:18.802063 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddfb42e-34e5-4a34-b501-63dc8850263f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:18.819074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 783/1562] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:18.819960 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52521d76-1eb8-4ede-a687-431b0f192bd4 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:18.829296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52521d76-1eb8-4ede-a687-431b0f192bd4 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:18.834251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd200d-d90b-44be-aa81-3e8e440834e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:18.834828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd200d-d90b-44be-aa81-3e8e440834e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:18.835113 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd200d-d90b-44be-aa81-3e8e440834e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:18.835424 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd200d-d90b-44be-aa81-3e8e440834e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:18.836064 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd200d-d90b-44be-aa81-3e8e440834e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:41:18.836372 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd200d-d90b-44be-aa81-3e8e440834e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:18.837949 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd200d-d90b-44be-aa81-3e8e440834e1 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:18.850200 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 784/1563] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:41:18.850963 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd200d-d90b-44be-aa81-3e8e440834e1 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:18.860514 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd200d-d90b-44be-aa81-3e8e440834e1 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:18.866175 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de500727-24e2-4472-b62f-9028595bb2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:18.866748 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de500727-24e2-4472-b62f-9028595bb2a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:18.867065 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de500727-24e2-4472-b62f-9028595bb2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:18.867465 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de500727-24e2-4472-b62f-9028595bb2a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:18.868122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de500727-24e2-4472-b62f-9028595bb2a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:18.870025 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de500727-24e2-4472-b62f-9028595bb2a0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:18.922090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 780/1564] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:41:18 2022] POST /identity/v3/projects => generated 393 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:18.923140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19662c-3fff-4929-87e6-c8cae35e7489 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:18.923629 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19662c-3fff-4929-87e6-c8cae35e7489 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:18.923934 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19662c-3fff-4929-87e6-c8cae35e7489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:18.943312 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-bd19662c-3fff-4929-87e6-c8cae35e7489 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:18.944539 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bd19662c-3fff-4929-87e6-c8cae35e7489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:19.002810 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 781/1565] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:19.004233 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19662c-3fff-4929-87e6-c8cae35e7489 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:19.012371 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 785/1566] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:41:18 2022] POST /identity/v3/projects => generated 379 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:19.012371 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dac8ca-9e74-4c53-866f-31f0cad8546e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:19.012371 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dac8ca-9e74-4c53-866f-31f0cad8546e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:19.012371 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dac8ca-9e74-4c53-866f-31f0cad8546e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:19.016813 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19662c-3fff-4929-87e6-c8cae35e7489 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:19.030653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:19.039983 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f2dac8ca-9e74-4c53-866f-31f0cad8546e None None] MFA Rules not processed for user `5ce0e999ef0b4b26ac24d703661d51a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:19.041210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f2dac8ca-9e74-4c53-866f-31f0cad8546e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:19.073495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed761b70-5222-49d3-ade2-c3a790087d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:19.074135 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed761b70-5222-49d3-ade2-c3a790087d7b admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:19.074491 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed761b70-5222-49d3-ade2-c3a790087d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:19.074842 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed761b70-5222-49d3-ade2-c3a790087d7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:19.075502 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed761b70-5222-49d3-ade2-c3a790087d7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:19.077406 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed761b70-5222-49d3-ade2-c3a790087d7b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:19.105777 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 786/1567] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:19.107045 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d639b-fd2b-40ef-ac3a-b39ecb017c72 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:19.107351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d639b-fd2b-40ef-ac3a-b39ecb017c72 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:19.107629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d639b-fd2b-40ef-ac3a-b39ecb017c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:19.130460 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a61d639b-fd2b-40ef-ac3a-b39ecb017c72 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:19.131741 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a61d639b-fd2b-40ef-ac3a-b39ecb017c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:19.140212 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 787/1568] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:19.140212 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d639b-fd2b-40ef-ac3a-b39ecb017c72 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:19.150500 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 782/1569] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:41:19 2022] POST /identity/v3/projects => generated 367 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:19.150500 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed761b70-5222-49d3-ade2-c3a790087d7b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:19.153250 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d639b-fd2b-40ef-ac3a-b39ecb017c72 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:19.211395 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed761b70-5222-49d3-ade2-c3a790087d7b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:19.214136 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62248e19-a75e-416f-b1a5-f99f17288e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:19.214752 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62248e19-a75e-416f-b1a5-f99f17288e1f admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:19.215033 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62248e19-a75e-416f-b1a5-f99f17288e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:19.215345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62248e19-a75e-416f-b1a5-f99f17288e1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:19.215951 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d5af5-b58b-4206-8111-8edd1499d8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:19.216085 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62248e19-a75e-416f-b1a5-f99f17288e1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:41:19.216278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62248e19-a75e-416f-b1a5-f99f17288e1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:19.216555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d5af5-b58b-4206-8111-8edd1499d8d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:19.216837 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d5af5-b58b-4206-8111-8edd1499d8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:19.217111 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d5af5-b58b-4206-8111-8edd1499d8d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:19.217732 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d5af5-b58b-4206-8111-8edd1499d8d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:19.217956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62248e19-a75e-416f-b1a5-f99f17288e1f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:19.219642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d5af5-b58b-4206-8111-8edd1499d8d7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:19.225465 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:19.229532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 788/1570] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:41:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:41:19.230319 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62248e19-a75e-416f-b1a5-f99f17288e1f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:19.244445 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62248e19-a75e-416f-b1a5-f99f17288e1f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:19.249098 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0bbbe-90e1-42c6-b3c3-fbbaf761c775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:19.255075 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0bbbe-90e1-42c6-b3c3-fbbaf761c775 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:19.255075 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0bbbe-90e1-42c6-b3c3-fbbaf761c775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:19.255075 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0bbbe-90e1-42c6-b3c3-fbbaf761c775 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:19.255075 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0bbbe-90e1-42c6-b3c3-fbbaf761c775 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:19.255075 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0bbbe-90e1-42c6-b3c3-fbbaf761c775 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:19.310641 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 783/1571] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:41:19 2022] POST /identity/v3/projects => generated 365 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:19.312501 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d5af5-b58b-4206-8111-8edd1499d8d7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:19.342939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 789/1572] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:41:19 2022] POST /identity/v3/users => generated 297 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:19.342939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0bbbe-90e1-42c6-b3c3-fbbaf761c775 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:19.348887 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0bbbe-90e1-42c6-b3c3-fbbaf761c775 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:19.379884 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d5af5-b58b-4206-8111-8edd1499d8d7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:19.385073 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574b9b6-c4a3-4432-a48b-17c5e8120f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:19.385798 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574b9b6-c4a3-4432-a48b-17c5e8120f3e admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:19.386080 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574b9b6-c4a3-4432-a48b-17c5e8120f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:19.386351 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574b9b6-c4a3-4432-a48b-17c5e8120f3e admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:19.387017 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574b9b6-c4a3-4432-a48b-17c5e8120f3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:41:19.387356 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574b9b6-c4a3-4432-a48b-17c5e8120f3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:19.388990 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574b9b6-c4a3-4432-a48b-17c5e8120f3e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:19.401241 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 784/1573] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:41:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:41:19.402468 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cb405-86c1-4ebb-9224-b6024c2300c1 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:19.402754 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cb405-86c1-4ebb-9224-b6024c2300c1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:19.408798 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cb405-86c1-4ebb-9224-b6024c2300c1 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:19.408798 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 785/1574] 10.222.0.49 () {54 vars in 962 bytes} [Wed Aug 10 12:41:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:41:19.408798 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324cb405-86c1-4ebb-9224-b6024c2300c1 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:19.409101 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8317a1e6-29f9-4aec-954f-91f0f69d8c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:19.409488 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317a1e6-29f9-4aec-954f-91f0f69d8c5b admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:19.409863 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317a1e6-29f9-4aec-954f-91f0f69d8c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:19.410162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317a1e6-29f9-4aec-954f-91f0f69d8c5b admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:19.410947 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8317a1e6-29f9-4aec-954f-91f0f69d8c5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:19.412980 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8317a1e6-29f9-4aec-954f-91f0f69d8c5b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:19.418455 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324cb405-86c1-4ebb-9224-b6024c2300c1 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:19.422973 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d896ac-3a50-40e8-aeed-9595228fccad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:19.423605 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d896ac-3a50-40e8-aeed-9595228fccad admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:19.425104 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d896ac-3a50-40e8-aeed-9595228fccad admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:19.425563 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d896ac-3a50-40e8-aeed-9595228fccad admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:19.428016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d896ac-3a50-40e8-aeed-9595228fccad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:41:19.428334 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d896ac-3a50-40e8-aeed-9595228fccad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:19.429943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d896ac-3a50-40e8-aeed-9595228fccad admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:19.443109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 786/1575] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:41:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:41:19.443913 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d896ac-3a50-40e8-aeed-9595228fccad admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:19.453500 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d896ac-3a50-40e8-aeed-9595228fccad admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:19.458545 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83182b4-1190-4a2a-aa02-b4433cc5f889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:19.459113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83182b4-1190-4a2a-aa02-b4433cc5f889 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:19.459425 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83182b4-1190-4a2a-aa02-b4433cc5f889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:19.459696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83182b4-1190-4a2a-aa02-b4433cc5f889 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:19.460297 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83182b4-1190-4a2a-aa02-b4433cc5f889 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:19.462148 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83182b4-1190-4a2a-aa02-b4433cc5f889 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:19.503732 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:19.510173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 790/1576] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:41:19 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:19.510952 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8317a1e6-29f9-4aec-954f-91f0f69d8c5b admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:19.520737 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8317a1e6-29f9-4aec-954f-91f0f69d8c5b admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:19.525561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b2da0-0e41-472a-84ef-6eefd466d6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:19.526131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b2da0-0e41-472a-84ef-6eefd466d6f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:19.526414 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b2da0-0e41-472a-84ef-6eefd466d6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:19.526689 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b2da0-0e41-472a-84ef-6eefd466d6f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:19.527338 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b2da0-0e41-472a-84ef-6eefd466d6f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:19.529265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b2da0-0e41-472a-84ef-6eefd466d6f3 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:19.566094 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 787/1577] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:41:19 2022] POST /identity/v3/users => generated 284 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:19.567731 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83182b4-1190-4a2a-aa02-b4433cc5f889 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:19.577145 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83182b4-1190-4a2a-aa02-b4433cc5f889 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:19.581796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f04fed-f5f5-4c9e-9d6d-bded2f328b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:19.582486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f04fed-f5f5-4c9e-9d6d-bded2f328b9a admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:19.582978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f04fed-f5f5-4c9e-9d6d-bded2f328b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:19.583320 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f04fed-f5f5-4c9e-9d6d-bded2f328b9a admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:19.583926 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f04fed-f5f5-4c9e-9d6d-bded2f328b9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:19.585811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f04fed-f5f5-4c9e-9d6d-bded2f328b9a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:19.653532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 791/1578] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:41:19 2022] POST /identity/v3/projects => generated 389 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:19.654093 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b2da0-0e41-472a-84ef-6eefd466d6f3 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:19.682641 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 788/1579] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:41:19 2022] POST /identity/v3/users => generated 283 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:19.683546 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f04fed-f5f5-4c9e-9d6d-bded2f328b9a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:19.694106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f04fed-f5f5-4c9e-9d6d-bded2f328b9a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:19.721248 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b2da0-0e41-472a-84ef-6eefd466d6f3 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:19.727111 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0c046-fcdb-4f26-98af-ac188dd0e81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:19.727754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0c046-fcdb-4f26-98af-ac188dd0e81f admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:19.728032 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0c046-fcdb-4f26-98af-ac188dd0e81f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:19.728306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0c046-fcdb-4f26-98af-ac188dd0e81f admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:19.728939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0c046-fcdb-4f26-98af-ac188dd0e81f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:19.730508 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0c046-fcdb-4f26-98af-ac188dd0e81f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:19.741494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 792/1580] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:19 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:19.742767 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf2a43-110c-488d-9fde-8a58de487789 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:19.743057 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf2a43-110c-488d-9fde-8a58de487789 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:19.743356 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf2a43-110c-488d-9fde-8a58de487789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:19.752476 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d368f-c55e-4a78-8dcd-97718991f13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:19.753086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d368f-c55e-4a78-8dcd-97718991f13e admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:19.753364 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d368f-c55e-4a78-8dcd-97718991f13e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:19.753635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d368f-c55e-4a78-8dcd-97718991f13e admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:19.754347 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d368f-c55e-4a78-8dcd-97718991f13e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:19.756410 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d368f-c55e-4a78-8dcd-97718991f13e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:19.781993 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f6bf2a43-110c-488d-9fde-8a58de487789 None None] MFA Rules not processed for user `eb3f6deffb54467f8b923956ed2a6acf`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:19.784180 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f6bf2a43-110c-488d-9fde-8a58de487789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:19.841120 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 789/1581] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:41:19 2022] POST /identity/v3/projects => generated 385 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:19.841120 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d368f-c55e-4a78-8dcd-97718991f13e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:19.852242 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 793/1582] 10.222.0.49 () {58 vars in 1059 bytes} [Wed Aug 10 12:41:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:19.853005 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bf2a43-110c-488d-9fde-8a58de487789 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:19.859592 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:19.863977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bf2a43-110c-488d-9fde-8a58de487789 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:19.903503 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d368f-c55e-4a78-8dcd-97718991f13e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:19.908566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72fcb51-f056-4adf-87b0-b0becee81a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:19.909150 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72fcb51-f056-4adf-87b0-b0becee81a95 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:19.909431 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72fcb51-f056-4adf-87b0-b0becee81a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:19.909703 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72fcb51-f056-4adf-87b0-b0becee81a95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:19.910319 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72fcb51-f056-4adf-87b0-b0becee81a95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:19.912469 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:19.912820 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72fcb51-f056-4adf-87b0-b0becee81a95 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:19.928399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513dccf-d09a-48a8-9bb3-cbb226ca5e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:19.929650 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513dccf-d09a-48a8-9bb3-cbb226ca5e66 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:19.929650 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513dccf-d09a-48a8-9bb3-cbb226ca5e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:19.929885 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513dccf-d09a-48a8-9bb3-cbb226ca5e66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:19.930528 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f513dccf-d09a-48a8-9bb3-cbb226ca5e66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:19.932293 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f513dccf-d09a-48a8-9bb3-cbb226ca5e66 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:19.942283 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 794/1583] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:19 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:19.943081 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513dccf-d09a-48a8-9bb3-cbb226ca5e66 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:19.952848 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513dccf-d09a-48a8-9bb3-cbb226ca5e66 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:19.957907 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fdcc22-4e7c-4e4f-bbf5-f7622134032f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:19.958476 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdcc22-4e7c-4e4f-bbf5-f7622134032f admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:19.958754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdcc22-4e7c-4e4f-bbf5-f7622134032f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:19.959025 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdcc22-4e7c-4e4f-bbf5-f7622134032f admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:19.959668 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fdcc22-4e7c-4e4f-bbf5-f7622134032f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:19.961218 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fdcc22-4e7c-4e4f-bbf5-f7622134032f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:19.970741 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 795/1584] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:19.971508 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fdcc22-4e7c-4e4f-bbf5-f7622134032f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:19.983596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fdcc22-4e7c-4e4f-bbf5-f7622134032f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:19.990566 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a6653-2c02-44ba-aa07-fea07b09b147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:19.990566 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a6653-2c02-44ba-aa07-fea07b09b147 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:19.990566 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a6653-2c02-44ba-aa07-fea07b09b147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:19.990566 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a6653-2c02-44ba-aa07-fea07b09b147 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:19.991047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a6653-2c02-44ba-aa07-fea07b09b147 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:19.992150 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a6653-2c02-44ba-aa07-fea07b09b147 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:20.014450 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 790/1585] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:41:19 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:20.015467 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72fcb51-f056-4adf-87b0-b0becee81a95 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:20.076188 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72fcb51-f056-4adf-87b0-b0becee81a95 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:20.080835 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d46ea9-8cf8-4b9f-b951-64cae33b8045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:20.081426 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d46ea9-8cf8-4b9f-b951-64cae33b8045 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:20.081708 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d46ea9-8cf8-4b9f-b951-64cae33b8045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:20.081980 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d46ea9-8cf8-4b9f-b951-64cae33b8045 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:20.083143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d46ea9-8cf8-4b9f-b951-64cae33b8045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:20.084845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d46ea9-8cf8-4b9f-b951-64cae33b8045 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:20.094381 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 791/1586] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:20.095155 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d46ea9-8cf8-4b9f-b951-64cae33b8045 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:20.105340 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d46ea9-8cf8-4b9f-b951-64cae33b8045 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:20.106556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 796/1587] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:41:19 2022] POST /identity/v3/users => generated 295 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:20.110639 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f865f-bfcd-42c8-be37-8ac26ddc99c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:20.110840 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a6653-2c02-44ba-aa07-fea07b09b147 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:20.114200 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f865f-bfcd-42c8-be37-8ac26ddc99c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:20.114200 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f865f-bfcd-42c8-be37-8ac26ddc99c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:20.114200 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f865f-bfcd-42c8-be37-8ac26ddc99c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:20.114200 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f865f-bfcd-42c8-be37-8ac26ddc99c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:20.114633 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f865f-bfcd-42c8-be37-8ac26ddc99c5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:20.120971 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a6653-2c02-44ba-aa07-fea07b09b147 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:20.124984 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 792/1588] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:20.125645 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f865f-bfcd-42c8-be37-8ac26ddc99c5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:20.128768 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018f2ba-f7d4-4c55-968d-7c04e5228b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:20.130616 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018f2ba-f7d4-4c55-968d-7c04e5228b20 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:20.131051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018f2ba-f7d4-4c55-968d-7c04e5228b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:20.131358 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018f2ba-f7d4-4c55-968d-7c04e5228b20 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:20.132042 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018f2ba-f7d4-4c55-968d-7c04e5228b20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:20.134023 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018f2ba-f7d4-4c55-968d-7c04e5228b20 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:20.197520 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f865f-bfcd-42c8-be37-8ac26ddc99c5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:20.202962 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d256d-3841-47b2-82fe-a72234250001 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:20.203599 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d256d-3841-47b2-82fe-a72234250001 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:20.203885 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d256d-3841-47b2-82fe-a72234250001 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:20.204162 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d256d-3841-47b2-82fe-a72234250001 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:20.218632 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 797/1589] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:41:20 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:20.219740 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018f2ba-f7d4-4c55-968d-7c04e5228b20 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:20.230286 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018f2ba-f7d4-4c55-968d-7c04e5228b20 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:20.230516 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:20.235275 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d0ffd-4c9c-4030-aed1-f51ac9cd77bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:20.235872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d0ffd-4c9c-4030-aed1-f51ac9cd77bd admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:20.236153 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d0ffd-4c9c-4030-aed1-f51ac9cd77bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:20.236430 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d0ffd-4c9c-4030-aed1-f51ac9cd77bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:20.237065 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07d0ffd-4c9c-4030-aed1-f51ac9cd77bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:20.239351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07d0ffd-4c9c-4030-aed1-f51ac9cd77bd admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:20.251315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 798/1590] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:20.255634 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d0ffd-4c9c-4030-aed1-f51ac9cd77bd admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:20.265408 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d0ffd-4c9c-4030-aed1-f51ac9cd77bd admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:20.266412 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d256d-3841-47b2-82fe-a72234250001 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:20.267990 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d256d-3841-47b2-82fe-a72234250001 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:20.270254 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0d06b-70d5-43ee-b832-ae72fd7a7619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:20.270843 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0d06b-70d5-43ee-b832-ae72fd7a7619 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:20.271120 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0d06b-70d5-43ee-b832-ae72fd7a7619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:20.271425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0d06b-70d5-43ee-b832-ae72fd7a7619 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:20.272077 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0d06b-70d5-43ee-b832-ae72fd7a7619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:20.273628 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0d06b-70d5-43ee-b832-ae72fd7a7619 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:20.283494 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 793/1591] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:20.286228 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 799/1592] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:20.286400 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d256d-3841-47b2-82fe-a72234250001 service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:20.286499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0d06b-70d5-43ee-b832-ae72fd7a7619 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:20.296516 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0d06b-70d5-43ee-b832-ae72fd7a7619 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:20.297034 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d256d-3841-47b2-82fe-a72234250001 service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:20.301759 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562774b-bf03-41fe-a651-0c7aba9f38ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:20.302446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562774b-bf03-41fe-a651-0c7aba9f38ef admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:20.302580 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23957a70-9ed7-4b66-bbbf-7cbfba764240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:20.302852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562774b-bf03-41fe-a651-0c7aba9f38ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:20.303267 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562774b-bf03-41fe-a651-0c7aba9f38ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:20.303444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23957a70-9ed7-4b66-bbbf-7cbfba764240 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:20.303852 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23957a70-9ed7-4b66-bbbf-7cbfba764240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:20.304079 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562774b-bf03-41fe-a651-0c7aba9f38ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:20.304379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23957a70-9ed7-4b66-bbbf-7cbfba764240 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:20.305073 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23957a70-9ed7-4b66-bbbf-7cbfba764240 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:20.305856 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562774b-bf03-41fe-a651-0c7aba9f38ef admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:20.307154 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23957a70-9ed7-4b66-bbbf-7cbfba764240 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:20.316453 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 800/1593] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:20.317561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562774b-bf03-41fe-a651-0c7aba9f38ef admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:20.327103 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562774b-bf03-41fe-a651-0c7aba9f38ef admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:20.331620 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d4352-624c-4be3-9e4d-ad8b809ef428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:20.336275 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d4352-624c-4be3-9e4d-ad8b809ef428 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:20.336275 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d4352-624c-4be3-9e4d-ad8b809ef428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:20.336275 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d4352-624c-4be3-9e4d-ad8b809ef428 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:20.336275 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43d4352-624c-4be3-9e4d-ad8b809ef428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:20.336275 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43d4352-624c-4be3-9e4d-ad8b809ef428 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:20.344633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 801/1594] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:20.348668 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d4352-624c-4be3-9e4d-ad8b809ef428 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:20.358163 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d4352-624c-4be3-9e4d-ad8b809ef428 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:20.362952 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44419c37-55a1-4151-a612-0a40d02c1754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:20.364929 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44419c37-55a1-4151-a612-0a40d02c1754 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:20.364929 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44419c37-55a1-4151-a612-0a40d02c1754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:20.364929 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44419c37-55a1-4151-a612-0a40d02c1754 admin admin] PATH_INFO: `/v3/projects/e41d5b8acbce44248a45678d1e4f6b71/users/39cb7e0c86b3460dbd17c58dc13ef0d9/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:20.379093 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44419c37-55a1-4151-a612-0a40d02c1754 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e41d5b8acbce44248a45678d1e4f6b71, user_id=39cb7e0c86b3460dbd17c58dc13ef0d9, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:20.381906 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44419c37-55a1-4151-a612-0a40d02c1754 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:20.422107 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 794/1595] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:41:20 2022] POST /identity/v3/users => generated 289 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:20.422107 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23957a70-9ed7-4b66-bbbf-7cbfba764240 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:20.431025 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23957a70-9ed7-4b66-bbbf-7cbfba764240 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:20.435931 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3507a-4b95-4f1b-9b10-017370739a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:20.436521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3507a-4b95-4f1b-9b10-017370739a4c admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:20.436799 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3507a-4b95-4f1b-9b10-017370739a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:20.437090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3507a-4b95-4f1b-9b10-017370739a4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:20.437791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c3507a-4b95-4f1b-9b10-017370739a4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:20.439497 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c3507a-4b95-4f1b-9b10-017370739a4c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:20.452887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 795/1596] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:20.452887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3507a-4b95-4f1b-9b10-017370739a4c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:20.459798 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3507a-4b95-4f1b-9b10-017370739a4c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:20.464358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48443f01-544b-4035-9a03-fd6991913f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:20.464984 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48443f01-544b-4035-9a03-fd6991913f7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:20.465265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48443f01-544b-4035-9a03-fd6991913f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:20.465545 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48443f01-544b-4035-9a03-fd6991913f7e admin admin] PATH_INFO: `/v3/projects/fefe2d12cf334ecbad2001b327cb65e8/users/51c598eb26614feaa0c31339b8135f08/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:20.478093 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 802/1597] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:41:20 2022] PUT /identity/v3/projects/e41d5b8acbce44248a45678d1e4f6b71/users/39cb7e0c86b3460dbd17c58dc13ef0d9/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:41:20.479295 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44419c37-55a1-4151-a612-0a40d02c1754 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:20.483642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48443f01-544b-4035-9a03-fd6991913f7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fefe2d12cf334ecbad2001b327cb65e8, user_id=51c598eb26614feaa0c31339b8135f08, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:20.485735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48443f01-544b-4035-9a03-fd6991913f7e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:20.543338 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44419c37-55a1-4151-a612-0a40d02c1754 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:20.547989 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a7309-8213-43f4-bc4e-293e8c8fe211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:20.548644 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a7309-8213-43f4-bc4e-293e8c8fe211 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:20.548922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a7309-8213-43f4-bc4e-293e8c8fe211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:20.549193 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a7309-8213-43f4-bc4e-293e8c8fe211 admin admin] PATH_INFO: `/v3/projects/54fe7893428b4bb09e7b3642e04c614f/users/0dfdede4a0184e1d9830a40adec0226b/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:20.563688 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413a7309-8213-43f4-bc4e-293e8c8fe211 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54fe7893428b4bb09e7b3642e04c614f, user_id=0dfdede4a0184e1d9830a40adec0226b, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:20.566352 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413a7309-8213-43f4-bc4e-293e8c8fe211 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:20.568068 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 796/1598] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:41:20 2022] PUT /identity/v3/projects/fefe2d12cf334ecbad2001b327cb65e8/users/51c598eb26614feaa0c31339b8135f08/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:41:20.571567 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48443f01-544b-4035-9a03-fd6991913f7e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:20.629902 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48443f01-544b-4035-9a03-fd6991913f7e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:20.634511 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37306991-369b-490d-9cbc-94845543460f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:20.635165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37306991-369b-490d-9cbc-94845543460f admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:20.635476 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37306991-369b-490d-9cbc-94845543460f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:20.635749 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37306991-369b-490d-9cbc-94845543460f admin admin] PATH_INFO: `/v3/projects/e9cc496fab1444c8b5ea5d98e48d3304/users/09c9a6a6f39c4b5fadd26ba4c53652ea/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:20.650405 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37306991-369b-490d-9cbc-94845543460f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9cc496fab1444c8b5ea5d98e48d3304, user_id=09c9a6a6f39c4b5fadd26ba4c53652ea, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:20.653058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37306991-369b-490d-9cbc-94845543460f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:20.657715 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 803/1599] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:41:20 2022] PUT /identity/v3/projects/54fe7893428b4bb09e7b3642e04c614f/users/0dfdede4a0184e1d9830a40adec0226b/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:41:20.660219 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a7309-8213-43f4-bc4e-293e8c8fe211 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:20.680958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:20.724184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a7309-8213-43f4-bc4e-293e8c8fe211 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:20.728924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c04e4b-d011-4e59-854e-38a674edc41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:20.731111 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c04e4b-d011-4e59-854e-38a674edc41d admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:20.731537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c04e4b-d011-4e59-854e-38a674edc41d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:20.731819 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c04e4b-d011-4e59-854e-38a674edc41d admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:20.732466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c04e4b-d011-4e59-854e-38a674edc41d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:20.734061 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c04e4b-d011-4e59-854e-38a674edc41d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:20.740321 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 797/1600] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:41:20 2022] PUT /identity/v3/projects/e9cc496fab1444c8b5ea5d98e48d3304/users/09c9a6a6f39c4b5fadd26ba4c53652ea/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:41:20.741521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37306991-369b-490d-9cbc-94845543460f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:20.746210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 804/1601] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:20.746844 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c04e4b-d011-4e59-854e-38a674edc41d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:20.761037 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c04e4b-d011-4e59-854e-38a674edc41d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:20.808567 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:20.814275 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37306991-369b-490d-9cbc-94845543460f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:20.819102 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5b210-bcde-4011-946b-ba25d8dec4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:20.819740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5b210-bcde-4011-946b-ba25d8dec4aa admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:20.820020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5b210-bcde-4011-946b-ba25d8dec4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:20.820228 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983bf34b-e8a2-4089-8ed1-816d11f6d991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:20.820354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5b210-bcde-4011-946b-ba25d8dec4aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:20.820909 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5b210-bcde-4011-946b-ba25d8dec4aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:20.821018 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983bf34b-e8a2-4089-8ed1-816d11f6d991 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:20.821228 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983bf34b-e8a2-4089-8ed1-816d11f6d991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:20.821531 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983bf34b-e8a2-4089-8ed1-816d11f6d991 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:20.822193 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983bf34b-e8a2-4089-8ed1-816d11f6d991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:20.822522 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5b210-bcde-4011-946b-ba25d8dec4aa admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:20.823871 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983bf34b-e8a2-4089-8ed1-816d11f6d991 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:20.833045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 798/1602] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:20.834278 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04e6d4-a501-49fe-a5e4-70daa8d2f53c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:20.834747 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04e6d4-a501-49fe-a5e4-70daa8d2f53c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:20.835024 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04e6d4-a501-49fe-a5e4-70daa8d2f53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:20.837065 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 805/1603] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:20.837065 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb325f14-f124-4632-b950-1609f63e83fc None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:20.839872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb325f14-f124-4632-b950-1609f63e83fc None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:20.839872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb325f14-f124-4632-b950-1609f63e83fc None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:20.839872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 806/1604] 10.222.0.49 () {54 vars in 912 bytes} [Wed Aug 10 12:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:41:20.839872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42452cd5-22d2-46dc-a7c1-202f54ad0590 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:20.840176 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42452cd5-22d2-46dc-a7c1-202f54ad0590 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:20.840330 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42452cd5-22d2-46dc-a7c1-202f54ad0590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:20.854497 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5d04e6d4-a501-49fe-a5e4-70daa8d2f53c None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:20.855762 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5d04e6d4-a501-49fe-a5e4-70daa8d2f53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:20.862200 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-42452cd5-22d2-46dc-a7c1-202f54ad0590 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:20.863437 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-42452cd5-22d2-46dc-a7c1-202f54ad0590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:20.865661 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 799/1605] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:20.866866 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26cded-ff11-47e2-8b3c-e6feba05f6ac None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:20.867155 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26cded-ff11-47e2-8b3c-e6feba05f6ac None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:20.867455 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26cded-ff11-47e2-8b3c-e6feba05f6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:20.875014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 807/1606] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:20.875014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42452cd5-22d2-46dc-a7c1-202f54ad0590 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:20.882766 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42452cd5-22d2-46dc-a7c1-202f54ad0590 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:20.887980 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f480db8-2dc6-46fe-80b7-ddf9b4de5f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:20.888643 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f480db8-2dc6-46fe-80b7-ddf9b4de5f69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:20.889551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f480db8-2dc6-46fe-80b7-ddf9b4de5f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:20.889833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f480db8-2dc6-46fe-80b7-ddf9b4de5f69 admin admin] PATH_INFO: `/v3/projects/d6fe8b6c27e743eaad5fe6ffc8a40e81/users/70010d27b7f44bc699ea5ad8093fec3b/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:20.898330 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7a26cded-ff11-47e2-8b3c-e6feba05f6ac None None] MFA Rules not processed for user `0dfdede4a0184e1d9830a40adec0226b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:20.902696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7a26cded-ff11-47e2-8b3c-e6feba05f6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:20.906146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f480db8-2dc6-46fe-80b7-ddf9b4de5f69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6fe8b6c27e743eaad5fe6ffc8a40e81, user_id=70010d27b7f44bc699ea5ad8093fec3b, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:20.908921 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f480db8-2dc6-46fe-80b7-ddf9b4de5f69 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:20.964198 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 800/1607] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:20.965443 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee44d04-44ee-4147-8f02-d432841bb0f4 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:20.965737 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee44d04-44ee-4147-8f02-d432841bb0f4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:20.966050 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee44d04-44ee-4147-8f02-d432841bb0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:20.985188 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 808/1608] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:41:20 2022] PUT /identity/v3/projects/d6fe8b6c27e743eaad5fe6ffc8a40e81/users/70010d27b7f44bc699ea5ad8093fec3b/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:41:20.986339 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f480db8-2dc6-46fe-80b7-ddf9b4de5f69 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:20.998617 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-fee44d04-44ee-4147-8f02-d432841bb0f4 None None] MFA Rules not processed for user `09c9a6a6f39c4b5fadd26ba4c53652ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:20.999907 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fee44d04-44ee-4147-8f02-d432841bb0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:21.051869 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f480db8-2dc6-46fe-80b7-ddf9b4de5f69 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:21.056450 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82829d1e-bbca-4cbf-bb0b-c4675b255e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:21.057048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82829d1e-bbca-4cbf-bb0b-c4675b255e3f admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:21.057327 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82829d1e-bbca-4cbf-bb0b-c4675b255e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:21.057662 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82829d1e-bbca-4cbf-bb0b-c4675b255e3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:21.058296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82829d1e-bbca-4cbf-bb0b-c4675b255e3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:21.059909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82829d1e-bbca-4cbf-bb0b-c4675b255e3f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:21.066077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 801/1609] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:21.067510 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee44d04-44ee-4147-8f02-d432841bb0f4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:21.074687 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 809/1610] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:21.076421 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564157c1-6328-4eec-92cd-bc4c83378dd2 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:21.076934 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564157c1-6328-4eec-92cd-bc4c83378dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:21.077366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564157c1-6328-4eec-92cd-bc4c83378dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:21.078250 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee44d04-44ee-4147-8f02-d432841bb0f4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:21.083945 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ae050-1d47-4681-b555-f26f0e5d706d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:21.084595 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ae050-1d47-4681-b555-f26f0e5d706d admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:21.084872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ae050-1d47-4681-b555-f26f0e5d706d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:21.085142 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ae050-1d47-4681-b555-f26f0e5d706d admin admin] PATH_INFO: `/v3/projects/a2857015013f466d95f3a9037925c6fc/users/570c2a77b862411083b9444f8292192b/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:21.098458 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-564157c1-6328-4eec-92cd-bc4c83378dd2 None None] MFA Rules not processed for user `eb3f6deffb54467f8b923956ed2a6acf`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:21.099671 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-564157c1-6328-4eec-92cd-bc4c83378dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:21.100348 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ae050-1d47-4681-b555-f26f0e5d706d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2857015013f466d95f3a9037925c6fc, user_id=570c2a77b862411083b9444f8292192b, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:21.102981 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ae050-1d47-4681-b555-f26f0e5d706d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:21.109020 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:21.165568 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 810/1611] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 10 12:41:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:21.166427 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564157c1-6328-4eec-92cd-bc4c83378dd2 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:21.167905 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-564157c1-6328-4eec-92cd-bc4c83378dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:21.181536 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 802/1612] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:41:21 2022] PUT /identity/v3/projects/a2857015013f466d95f3a9037925c6fc/users/570c2a77b862411083b9444f8292192b/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:41:21.181536 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ae050-1d47-4681-b555-f26f0e5d706d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:21.224304 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:21.230387 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564157c1-6328-4eec-92cd-bc4c83378dd2 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:21.234676 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393807c2-10e7-4a7e-9dcf-92e29cb0b215 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:21.240961 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393807c2-10e7-4a7e-9dcf-92e29cb0b215 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:21.240961 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393807c2-10e7-4a7e-9dcf-92e29cb0b215 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:21.240961 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393807c2-10e7-4a7e-9dcf-92e29cb0b215 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:21.241262 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ae050-1d47-4681-b555-f26f0e5d706d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:21.241402 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceef6a6-4d98-4430-ad81-96365817efd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:21.242046 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceef6a6-4d98-4430-ad81-96365817efd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:21.242331 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceef6a6-4d98-4430-ad81-96365817efd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:21.242609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceef6a6-4d98-4430-ad81-96365817efd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:21.297300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393807c2-10e7-4a7e-9dcf-92e29cb0b215 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:21.298743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393807c2-10e7-4a7e-9dcf-92e29cb0b215 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:21.306561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceef6a6-4d98-4430-ad81-96365817efd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:21.308040 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceef6a6-4d98-4430-ad81-96365817efd2 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:21.309002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 811/1613] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:21.309887 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393807c2-10e7-4a7e-9dcf-92e29cb0b215 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:21.319301 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 803/1614] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:21.320258 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cadbe3-4919-4940-b87f-c57602ae96db None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:21.320530 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cadbe3-4919-4940-b87f-c57602ae96db None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:21.320803 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cadbe3-4919-4940-b87f-c57602ae96db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:21.342106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e8cadbe3-4919-4940-b87f-c57602ae96db None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:21.344920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e8cadbe3-4919-4940-b87f-c57602ae96db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:21.352173 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 804/1615] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:21.356257 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cadbe3-4919-4940-b87f-c57602ae96db None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:21.366323 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cadbe3-4919-4940-b87f-c57602ae96db None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:21.370674 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393807c2-10e7-4a7e-9dcf-92e29cb0b215 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:21.370942 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0754e-f45a-4a0e-aadc-3e12eec6323b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:21.371567 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0754e-f45a-4a0e-aadc-3e12eec6323b admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:21.371872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0754e-f45a-4a0e-aadc-3e12eec6323b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:21.372111 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0754e-f45a-4a0e-aadc-3e12eec6323b admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:21.372728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee0754e-f45a-4a0e-aadc-3e12eec6323b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:21.373393 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61db029-10ce-46f8-ba82-6420bf834ace service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:21.374005 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61db029-10ce-46f8-ba82-6420bf834ace service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:21.374283 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61db029-10ce-46f8-ba82-6420bf834ace service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:21.374560 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61db029-10ce-46f8-ba82-6420bf834ace service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:21.374679 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee0754e-f45a-4a0e-aadc-3e12eec6323b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:21.383620 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:21.437115 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61db029-10ce-46f8-ba82-6420bf834ace service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:21.438689 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61db029-10ce-46f8-ba82-6420bf834ace service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:21.450492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 812/1616] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:21.451292 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61db029-10ce-46f8-ba82-6420bf834ace service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:21.464835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61db029-10ce-46f8-ba82-6420bf834ace service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:21.470809 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb245d-ac6f-4a2b-a89c-612009f4a131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:21.471480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb245d-ac6f-4a2b-a89c-612009f4a131 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:21.471814 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb245d-ac6f-4a2b-a89c-612009f4a131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:21.472146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb245d-ac6f-4a2b-a89c-612009f4a131 admin admin] PATH_INFO: `/v3/projects/9a40b36dd9944cb88fd54181f7f29685/users/c0ada1accf1742b393e8a12c60074b9e/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:21.491676 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdb245d-ac6f-4a2b-a89c-612009f4a131 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a40b36dd9944cb88fd54181f7f29685, user_id=c0ada1accf1742b393e8a12c60074b9e, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:21.494460 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdb245d-ac6f-4a2b-a89c-612009f4a131 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:21.523579 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 805/1617] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:41:21 2022] POST /identity/v3/projects => generated 365 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:21.524848 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bcda3-e77c-46f9-a63f-f1a2802e563e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:21.528112 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bcda3-e77c-46f9-a63f-f1a2802e563e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:21.528112 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bcda3-e77c-46f9-a63f-f1a2802e563e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:21.544717 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ab7bcda3-e77c-46f9-a63f-f1a2802e563e None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:21.545920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ab7bcda3-e77c-46f9-a63f-f1a2802e563e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:21.615609 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 813/1618] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:41:21 2022] PUT /identity/v3/projects/9a40b36dd9944cb88fd54181f7f29685/users/c0ada1accf1742b393e8a12c60074b9e/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:41:21.617267 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb245d-ac6f-4a2b-a89c-612009f4a131 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:21.656304 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 806/1619] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:21.657234 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bcda3-e77c-46f9-a63f-f1a2802e563e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:21.709493 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb245d-ac6f-4a2b-a89c-612009f4a131 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:21.715754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cda027-f1a3-4c9d-bd80-270a5a511fc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:21.716455 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cda027-f1a3-4c9d-bd80-270a5a511fc1 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:21.718962 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cda027-f1a3-4c9d-bd80-270a5a511fc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:21.718962 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cda027-f1a3-4c9d-bd80-270a5a511fc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:21.764284 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bcda3-e77c-46f9-a63f-f1a2802e563e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:21.781934 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:21.820081 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cda027-f1a3-4c9d-bd80-270a5a511fc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:21.821633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cda027-f1a3-4c9d-bd80-270a5a511fc1 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:21.824637 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189eb93f-9fe3-4784-932d-6168793daaa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:21.825338 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189eb93f-9fe3-4784-932d-6168793daaa6 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:21.825687 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189eb93f-9fe3-4784-932d-6168793daaa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:21.826025 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189eb93f-9fe3-4784-932d-6168793daaa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:21.833251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 814/1620] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:21.834179 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cda027-f1a3-4c9d-bd80-270a5a511fc1 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:21.866390 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:21.915426 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cda027-f1a3-4c9d-bd80-270a5a511fc1 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:21.921736 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591e006-e0bc-48f0-b7c0-9bc664030a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:21.922770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591e006-e0bc-48f0-b7c0-9bc664030a31 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:21.923476 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591e006-e0bc-48f0-b7c0-9bc664030a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:21.923762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591e006-e0bc-48f0-b7c0-9bc664030a31 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:21.924405 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591e006-e0bc-48f0-b7c0-9bc664030a31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:21.926602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591e006-e0bc-48f0-b7c0-9bc664030a31 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:21.934524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189eb93f-9fe3-4784-932d-6168793daaa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:21.936671 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189eb93f-9fe3-4784-932d-6168793daaa6 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:21.948648 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 807/1621] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:21.950093 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1750382-fcc8-482c-a0a3-068f70796338 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:21.950387 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1750382-fcc8-482c-a0a3-068f70796338 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:21.950718 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1750382-fcc8-482c-a0a3-068f70796338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:22.009896 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a1750382-fcc8-482c-a0a3-068f70796338 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:22.014512 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a1750382-fcc8-482c-a0a3-068f70796338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:22.023820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 815/1622] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:41:21 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:22.024787 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591e006-e0bc-48f0-b7c0-9bc664030a31 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:22.028079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 808/1623] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:22.029528 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1750382-fcc8-482c-a0a3-068f70796338 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:22.059488 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591e006-e0bc-48f0-b7c0-9bc664030a31 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:22.067369 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1750382-fcc8-482c-a0a3-068f70796338 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:22.071297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601969b8-4578-48d6-8aa0-017c2530b3b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:22.073498 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601969b8-4578-48d6-8aa0-017c2530b3b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:22.073785 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601969b8-4578-48d6-8aa0-017c2530b3b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:22.074211 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601969b8-4578-48d6-8aa0-017c2530b3b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:22.080446 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3df7cf-bdc0-4b0c-98a6-21df88fc0f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:22.081077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3df7cf-bdc0-4b0c-98a6-21df88fc0f64 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:22.081338 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3df7cf-bdc0-4b0c-98a6-21df88fc0f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:22.081607 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3df7cf-bdc0-4b0c-98a6-21df88fc0f64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:22.082247 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3df7cf-bdc0-4b0c-98a6-21df88fc0f64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:22.086837 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3df7cf-bdc0-4b0c-98a6-21df88fc0f64 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:22.095706 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601969b8-4578-48d6-8aa0-017c2530b3b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:22.097240 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601969b8-4578-48d6-8aa0-017c2530b3b6 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:22.110801 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 816/1624] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:22.111824 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601969b8-4578-48d6-8aa0-017c2530b3b6 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:22.122999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601969b8-4578-48d6-8aa0-017c2530b3b6 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:22.127592 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 809/1625] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:22.131361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ea610-2b5b-44db-ab1f-27d7f360d6a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:22.131929 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3df7cf-bdc0-4b0c-98a6-21df88fc0f64 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:22.132502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ea610-2b5b-44db-ab1f-27d7f360d6a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:22.133375 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ea610-2b5b-44db-ab1f-27d7f360d6a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:22.133944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ea610-2b5b-44db-ab1f-27d7f360d6a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:22.149243 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3df7cf-bdc0-4b0c-98a6-21df88fc0f64 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:22.153340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ea610-2b5b-44db-ab1f-27d7f360d6a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:22.155523 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ea610-2b5b-44db-ab1f-27d7f360d6a7 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:22.157305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1dee2-0b56-40ff-8e0b-dab13203d3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:22.159150 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1dee2-0b56-40ff-8e0b-dab13203d3e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:22.159457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1dee2-0b56-40ff-8e0b-dab13203d3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:22.159734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1dee2-0b56-40ff-8e0b-dab13203d3e6 admin admin] PATH_INFO: `/v3/projects/239a79a4e777461f826f2a66c544a964/users/a1af427b45c54ed1b1166e154a103891/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:22.171570 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 817/1626] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:22.179539 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b1dee2-0b56-40ff-8e0b-dab13203d3e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=239a79a4e777461f826f2a66c544a964, user_id=a1af427b45c54ed1b1166e154a103891, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:22.183885 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b1dee2-0b56-40ff-8e0b-dab13203d3e6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:22.306074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 810/1627] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:41:22 2022] PUT /identity/v3/projects/239a79a4e777461f826f2a66c544a964/users/a1af427b45c54ed1b1166e154a103891/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:41:22.309928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ea610-2b5b-44db-ab1f-27d7f360d6a7 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:22.319917 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ea610-2b5b-44db-ab1f-27d7f360d6a7 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:22.379121 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8866c-7623-430a-b5db-32b228592162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:22.379818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8866c-7623-430a-b5db-32b228592162 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:22.380179 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8866c-7623-430a-b5db-32b228592162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:22.380526 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8866c-7623-430a-b5db-32b228592162 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:22.381352 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b8866c-7623-430a-b5db-32b228592162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:22.383123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b8866c-7623-430a-b5db-32b228592162 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:22.402956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 818/1628] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:22.404227 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1dee2-0b56-40ff-8e0b-dab13203d3e6 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:22.415919 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1dee2-0b56-40ff-8e0b-dab13203d3e6 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:22.420495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2851f-54d5-4a57-a370-e74919ef576a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:22.421154 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2851f-54d5-4a57-a370-e74919ef576a admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:22.421434 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2851f-54d5-4a57-a370-e74919ef576a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:22.421710 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2851f-54d5-4a57-a370-e74919ef576a admin admin] PATH_INFO: `/v3/projects/239a79a4e777461f826f2a66c544a964/users/a1af427b45c54ed1b1166e154a103891/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:22.423666 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b2851f-54d5-4a57-a370-e74919ef576a admin admin] RBAC: Authorizing `identity:create_grant(project_id=239a79a4e777461f826f2a66c544a964, user_id=a1af427b45c54ed1b1166e154a103891, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:22.426378 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b2851f-54d5-4a57-a370-e74919ef576a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:22.446248 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 811/1629] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:41:22 2022] PUT /identity/v3/projects/239a79a4e777461f826f2a66c544a964/users/a1af427b45c54ed1b1166e154a103891/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:41:22.448194 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193423c6-42f8-4a05-8169-12d9f940eec1 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:22.448489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193423c6-42f8-4a05-8169-12d9f940eec1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:22.448943 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193423c6-42f8-4a05-8169-12d9f940eec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:22.514815 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-193423c6-42f8-4a05-8169-12d9f940eec1 None None] MFA Rules not processed for user `a1af427b45c54ed1b1166e154a103891`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:22.517123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-193423c6-42f8-4a05-8169-12d9f940eec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:22.648419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 819/1630] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:22 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:22.660422 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2851f-54d5-4a57-a370-e74919ef576a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:22.738425 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2851f-54d5-4a57-a370-e74919ef576a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:22.744548 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2258e05-52c6-4b86-ac42-f9e523017cdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:22.745336 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2258e05-52c6-4b86-ac42-f9e523017cdb service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:22.745672 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2258e05-52c6-4b86-ac42-f9e523017cdb service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:22.746002 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2258e05-52c6-4b86-ac42-f9e523017cdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:22.764776 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2258e05-52c6-4b86-ac42-f9e523017cdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:22.767472 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2258e05-52c6-4b86-ac42-f9e523017cdb service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:22.791767 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 812/1631] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:22 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:23.065096 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193423c6-42f8-4a05-8169-12d9f940eec1 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:23.092863 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193423c6-42f8-4a05-8169-12d9f940eec1 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:23.127306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d21e66-46da-4628-88be-5494042fb174 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:23.129835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d21e66-46da-4628-88be-5494042fb174 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:23.130201 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d21e66-46da-4628-88be-5494042fb174 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:23.131677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d21e66-46da-4628-88be-5494042fb174 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:23.166033 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:23.279032 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d21e66-46da-4628-88be-5494042fb174 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:23.280562 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d21e66-46da-4628-88be-5494042fb174 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:23.303135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 820/1632] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:23.362509 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2d2a6-2c76-4917-b3ea-fc404df42d10 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:23.363097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2d2a6-2c76-4917-b3ea-fc404df42d10 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:23.363097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2d2a6-2c76-4917-b3ea-fc404df42d10 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:23.364669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 813/1633] 10.222.0.49 () {54 vars in 962 bytes} [Wed Aug 10 12:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:41:23.371623 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52009e47-e8d4-40d8-a935-84a78c97e357 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:23.375287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52009e47-e8d4-40d8-a935-84a78c97e357 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:23.375287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52009e47-e8d4-40d8-a935-84a78c97e357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:23.397321 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-52009e47-e8d4-40d8-a935-84a78c97e357 None None] MFA Rules not processed for user `eb3f6deffb54467f8b923956ed2a6acf`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:23.398569 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-52009e47-e8d4-40d8-a935-84a78c97e357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:23.502008 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 821/1634] 10.222.0.49 () {58 vars in 1059 bytes} [Wed Aug 10 12:41:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:23.508587 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a17b8-4cc5-4680-9f6e-9e52c8085ef9 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:23.508882 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a17b8-4cc5-4680-9f6e-9e52c8085ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:23.509164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a17b8-4cc5-4680-9f6e-9e52c8085ef9 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:23.510856 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 814/1635] 10.222.0.49 () {54 vars in 912 bytes} [Wed Aug 10 12:41:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:41:23.516383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba03fb-37bd-4da7-a30e-d7d15207a51f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:23.516675 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba03fb-37bd-4da7-a30e-d7d15207a51f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:23.516958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba03fb-37bd-4da7-a30e-d7d15207a51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:23.537047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7dba03fb-37bd-4da7-a30e-d7d15207a51f None None] MFA Rules not processed for user `eb3f6deffb54467f8b923956ed2a6acf`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:23.538491 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7dba03fb-37bd-4da7-a30e-d7d15207a51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:23.548382 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 822/1636] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 10 12:41:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:23.561823 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a17b8-4cc5-4680-9f6e-9e52c8085ef9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:23.575701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a17b8-4cc5-4680-9f6e-9e52c8085ef9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:23.580610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8416373b-6f09-4279-985f-ef9f692cfa86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:23.581750 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416373b-6f09-4279-985f-ef9f692cfa86 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:23.582022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416373b-6f09-4279-985f-ef9f692cfa86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:23.582468 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416373b-6f09-4279-985f-ef9f692cfa86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:23.594290 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8416373b-6f09-4279-985f-ef9f692cfa86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:23.595882 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8416373b-6f09-4279-985f-ef9f692cfa86 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:23.608739 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 815/1637] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:32.670062 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e3db0-5664-4aa0-bee2-cf2c819c282c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:32.670533 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e3db0-5664-4aa0-bee2-cf2c819c282c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:32.670616 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e3db0-5664-4aa0-bee2-cf2c819c282c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:32.737914 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a74e3db0-5664-4aa0-bee2-cf2c819c282c None None] MFA Rules not processed for user `39cb7e0c86b3460dbd17c58dc13ef0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:32.737914 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a74e3db0-5664-4aa0-bee2-cf2c819c282c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:32.830452 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 823/1638] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:32 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:32.842470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae372c9f-d2ca-4c8e-98f2-3fb7bda5adf2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:32.847880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae372c9f-d2ca-4c8e-98f2-3fb7bda5adf2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:32.848767 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae372c9f-d2ca-4c8e-98f2-3fb7bda5adf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:32.857368 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:32.880297 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ae372c9f-d2ca-4c8e-98f2-3fb7bda5adf2 None None] MFA Rules not processed for user `39cb7e0c86b3460dbd17c58dc13ef0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:32.881781 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ae372c9f-d2ca-4c8e-98f2-3fb7bda5adf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:32.903488 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 816/1639] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:32 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:32.921336 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e3db0-5664-4aa0-bee2-cf2c819c282c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:32.966332 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e3db0-5664-4aa0-bee2-cf2c819c282c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:32.977816 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9bfe2-e5ce-4357-a30d-08850070176c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:32.978786 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9bfe2-e5ce-4357-a30d-08850070176c service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:32.979278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9bfe2-e5ce-4357-a30d-08850070176c service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:32.980407 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9bfe2-e5ce-4357-a30d-08850070176c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:33.003547 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9bfe2-e5ce-4357-a30d-08850070176c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:33.008682 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9bfe2-e5ce-4357-a30d-08850070176c service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:33.022042 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 824/1640] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:32 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:33.668537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d4888-ac48-4f23-b146-8001262b3ae9 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:33.670127 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d4888-ac48-4f23-b146-8001262b3ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:33.674584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d4888-ac48-4f23-b146-8001262b3ae9 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:33.674584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 817/1641] 10.222.0.49 () {54 vars in 912 bytes} [Wed Aug 10 12:41:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:41:33.679146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a00ada-082a-4e8d-a1cc-f6429bbecb16 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:33.679606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a00ada-082a-4e8d-a1cc-f6429bbecb16 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:33.679897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a00ada-082a-4e8d-a1cc-f6429bbecb16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:33.707869 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:33.788160 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e2a00ada-082a-4e8d-a1cc-f6429bbecb16 None None] MFA Rules not processed for user `5db814952f9f466abf6148b7a0db2173`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:33.791073 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e2a00ada-082a-4e8d-a1cc-f6429bbecb16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:33.905036 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 825/1642] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 10 12:41:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:33.922544 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d4888-ac48-4f23-b146-8001262b3ae9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:33.961359 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d4888-ac48-4f23-b146-8001262b3ae9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:33.974239 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7bc8ad-6308-4955-823a-ad6065e25f75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:33.975456 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7bc8ad-6308-4955-823a-ad6065e25f75 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:33.976252 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7bc8ad-6308-4955-823a-ad6065e25f75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:33.976707 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7bc8ad-6308-4955-823a-ad6065e25f75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:33.997677 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7bc8ad-6308-4955-823a-ad6065e25f75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:33.999661 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7bc8ad-6308-4955-823a-ad6065e25f75 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:34.010625 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 818/1643] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:35.070349 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da8c3b8-b37b-496f-8bc9-79edad2366d7 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:35.070867 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da8c3b8-b37b-496f-8bc9-79edad2366d7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:35.070993 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da8c3b8-b37b-496f-8bc9-79edad2366d7 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:35.072768 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 826/1644] 10.222.0.49 () {54 vars in 961 bytes} [Wed Aug 10 12:41:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:41:35.079000 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b8075-37bd-4747-a925-c62f77aba07c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:35.080005 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b8075-37bd-4747-a925-c62f77aba07c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:35.080423 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b8075-37bd-4747-a925-c62f77aba07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:35.107601 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911555f3-0094-413f-b9b3-dc6386a51144 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:35.108323 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911555f3-0094-413f-b9b3-dc6386a51144 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:35.108744 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911555f3-0094-413f-b9b3-dc6386a51144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:35.158795 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-008b8075-37bd-4747-a925-c62f77aba07c None None] MFA Rules not processed for user `3ff152e22374407eb72caf648a763107`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:35.162390 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-008b8075-37bd-4747-a925-c62f77aba07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:35.164964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-911555f3-0094-413f-b9b3-dc6386a51144 None None] MFA Rules not processed for user `39cb7e0c86b3460dbd17c58dc13ef0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:35.173270 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-911555f3-0094-413f-b9b3-dc6386a51144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:35.200879 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 827/1645] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:35 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:41:35.272817 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 819/1646] 10.222.0.49 () {58 vars in 1058 bytes} [Wed Aug 10 12:41:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:35.281100 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911555f3-0094-413f-b9b3-dc6386a51144 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:35.309903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911555f3-0094-413f-b9b3-dc6386a51144 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:35.365407 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d6a5ea-c693-467e-8fbd-97979ac33936 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff152e22374407eb72caf648a763107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:35.366064 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6a5ea-c693-467e-8fbd-97979ac33936 service placement] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:35.366348 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6a5ea-c693-467e-8fbd-97979ac33936 service placement] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:35.366626 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6a5ea-c693-467e-8fbd-97979ac33936 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:35.378406 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e7d6a5ea-c693-467e-8fbd-97979ac33936 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:35.441353 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d6a5ea-c693-467e-8fbd-97979ac33936 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:35.453687 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d6a5ea-c693-467e-8fbd-97979ac33936 service placement] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:35.509746 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 828/1647] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:36.090887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b8075-37bd-4747-a925-c62f77aba07c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:36.320414 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b8075-37bd-4747-a925-c62f77aba07c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:36.347011 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ade69-b24a-4825-928b-ac5ddc554a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:36.347343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ade69-b24a-4825-928b-ac5ddc554a7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:36.347630 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ade69-b24a-4825-928b-ac5ddc554a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:36.347903 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ade69-b24a-4825-928b-ac5ddc554a7c admin admin] PATH_INFO: `/v3/users/5ce0e999ef0b4b26ac24d703661d51a5` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:36.352798 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ade69-b24a-4825-928b-ac5ddc554a7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ce0e999ef0b4b26ac24d703661d51a5)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:36.354844 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ade69-b24a-4825-928b-ac5ddc554a7c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:36.957101 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e06b4c-c4b0-4b29-8fda-bc0e8ee751e5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:36.958647 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e06b4c-c4b0-4b29-8fda-bc0e8ee751e5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:36.959090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e06b4c-c4b0-4b29-8fda-bc0e8ee751e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:37.001921 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-79e06b4c-c4b0-4b29-8fda-bc0e8ee751e5 None None] MFA Rules not processed for user `51c598eb26614feaa0c31339b8135f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:37.003607 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-79e06b4c-c4b0-4b29-8fda-bc0e8ee751e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:37.176870 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 829/1648] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:41:37.184613 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d9386-7765-4462-9fa7-a36b893fa8fa None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:37.184959 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d9386-7765-4462-9fa7-a36b893fa8fa None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:37.185248 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d9386-7765-4462-9fa7-a36b893fa8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:37.209520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-252d9386-7765-4462-9fa7-a36b893fa8fa None None] MFA Rules not processed for user `51c598eb26614feaa0c31339b8135f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:37.210338 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-252d9386-7765-4462-9fa7-a36b893fa8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:37.217392 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 830/1649] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:41:37.283932 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d9386-7765-4462-9fa7-a36b893fa8fa None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:37.298107 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d9386-7765-4462-9fa7-a36b893fa8fa None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:37.310268 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c5bc6-3d47-4755-831f-5e8ef15fdd34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:37.310890 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c5bc6-3d47-4755-831f-5e8ef15fdd34 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:37.311169 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c5bc6-3d47-4755-831f-5e8ef15fdd34 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:37.311471 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c5bc6-3d47-4755-831f-5e8ef15fdd34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:37.322953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93c5bc6-3d47-4755-831f-5e8ef15fdd34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:37.324632 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93c5bc6-3d47-4755-831f-5e8ef15fdd34 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:37.338987 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 831/1650] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:41:37.767646 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa566162-08a5-4111-bec5-d36e4da18c0c None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:37.768150 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa566162-08a5-4111-bec5-d36e4da18c0c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:37.768430 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa566162-08a5-4111-bec5-d36e4da18c0c None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:37.770060 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 832/1651] 10.222.0.49 () {54 vars in 912 bytes} [Wed Aug 10 12:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:41:37.775294 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02157d27-d005-4710-bdfa-635a1473f234 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:37.775802 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02157d27-d005-4710-bdfa-635a1473f234 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:37.776312 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02157d27-d005-4710-bdfa-635a1473f234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:37.797743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-02157d27-d005-4710-bdfa-635a1473f234 None None] MFA Rules not processed for user `5db814952f9f466abf6148b7a0db2173`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:37.799053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-02157d27-d005-4710-bdfa-635a1473f234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:37.806562 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 833/1652] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 10 12:41:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:41:37.818808 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02157d27-d005-4710-bdfa-635a1473f234 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:37.827804 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02157d27-d005-4710-bdfa-635a1473f234 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:37.831450 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af16a8b-e0d8-4e35-b563-eed830728922 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:37.832052 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af16a8b-e0d8-4e35-b563-eed830728922 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:37.832859 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af16a8b-e0d8-4e35-b563-eed830728922 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:37.833137 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af16a8b-e0d8-4e35-b563-eed830728922 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:37.840321 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af16a8b-e0d8-4e35-b563-eed830728922 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:37.841729 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af16a8b-e0d8-4e35-b563-eed830728922 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:37.850248 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 834/1653] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:41:38.457184 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8b0ade69-b24a-4825-928b-ac5ddc554a7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ce0e999ef0b4b26ac24d703661d51a5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:38.556572 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8b0ade69-b24a-4825-928b-ac5ddc554a7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ce0e999ef0b4b26ac24d703661d51a5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:38.557642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8b0ade69-b24a-4825-928b-ac5ddc554a7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ce0e999ef0b4b26ac24d703661d51a5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:38.559230 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8b0ade69-b24a-4825-928b-ac5ddc554a7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ce0e999ef0b4b26ac24d703661d51a5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:38.582914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8b0ade69-b24a-4825-928b-ac5ddc554a7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ce0e999ef0b4b26ac24d703661d51a5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:38.667136 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:38.692792 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 820/1654] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:41:36 2022] DELETE /identity/v3/users/5ce0e999ef0b4b26ac24d703661d51a5 => generated 0 bytes in 2602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:41:38.697168 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6689e31-c726-4ab8-b73d-dc69f2c062c4 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:38.697673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6689e31-c726-4ab8-b73d-dc69f2c062c4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:38.697842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6689e31-c726-4ab8-b73d-dc69f2c062c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:38.752521 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d6689e31-c726-4ab8-b73d-dc69f2c062c4 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:38.762545 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d6689e31-c726-4ab8-b73d-dc69f2c062c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:38.936318 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:39.015652 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 835/1655] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:39.094969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ade69-b24a-4825-928b-ac5ddc554a7c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:39.097025 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8b0ade69-b24a-4825-928b-ac5ddc554a7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:39.286996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ade69-b24a-4825-928b-ac5ddc554a7c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:39.299993 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043d1e4-d19a-4b72-95a3-0709553f8602 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:39.302039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043d1e4-d19a-4b72-95a3-0709553f8602 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:39.307877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043d1e4-d19a-4b72-95a3-0709553f8602 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:39.307877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043d1e4-d19a-4b72-95a3-0709553f8602 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:39.326800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043d1e4-d19a-4b72-95a3-0709553f8602 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:39.328822 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043d1e4-d19a-4b72-95a3-0709553f8602 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:39.358388 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 821/1656] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:40.537426 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a9c95-99f6-44df-bec6-0a5ee7245ee1 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:40.537996 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a9c95-99f6-44df-bec6-0a5ee7245ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:40.537996 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a9c95-99f6-44df-bec6-0a5ee7245ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:40.697987 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-df4a9c95-99f6-44df-bec6-0a5ee7245ee1 None None] MFA Rules not processed for user `70010d27b7f44bc699ea5ad8093fec3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:40.699807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-df4a9c95-99f6-44df-bec6-0a5ee7245ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:40.894061 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 836/1657] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:40.916452 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043d1e4-d19a-4b72-95a3-0709553f8602 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:40.917907 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e043d1e4-d19a-4b72-95a3-0709553f8602 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:41.023890 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a9c95-99f6-44df-bec6-0a5ee7245ee1 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:41.025752 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-df4a9c95-99f6-44df-bec6-0a5ee7245ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:41.063991 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a9c95-99f6-44df-bec6-0a5ee7245ee1 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:41.086213 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e16309a-b2ce-4628-8599-da87c49c5332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:41.090621 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16309a-b2ce-4628-8599-da87c49c5332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:41.090621 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16309a-b2ce-4628-8599-da87c49c5332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:41.090621 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16309a-b2ce-4628-8599-da87c49c5332 admin admin] PATH_INFO: `/v3/projects/578ea63d03fc48c881899b6482d954e0` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:41.090621 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e16309a-b2ce-4628-8599-da87c49c5332 admin admin] RBAC: Authorizing `identity:delete_project(project_id=578ea63d03fc48c881899b6482d954e0)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:41.091011 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e16309a-b2ce-4628-8599-da87c49c5332 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:41.140636 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043d1e4-d19a-4b72-95a3-0709553f8602 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:41.162282 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc2b8e-8487-48bb-b1bb-99982ad1bf7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:41.163945 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc2b8e-8487-48bb-b1bb-99982ad1bf7d service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:41.163945 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc2b8e-8487-48bb-b1bb-99982ad1bf7d service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:41.164802 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc2b8e-8487-48bb-b1bb-99982ad1bf7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:41.191147 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdc2b8e-8487-48bb-b1bb-99982ad1bf7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:41.193967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdc2b8e-8487-48bb-b1bb-99982ad1bf7d service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:41.243056 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 822/1658] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:40 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:41.310108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9e16309a-b2ce-4628-8599-da87c49c5332 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '578ea63d03fc48c881899b6482d954e0', 'request_id': 'req-9e16309a-b2ce-4628-8599-da87c49c5332'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:41.338471 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9e16309a-b2ce-4628-8599-da87c49c5332 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '578ea63d03fc48c881899b6482d954e0', 'request_id': 'req-9e16309a-b2ce-4628-8599-da87c49c5332'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:41.475003 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fa294-8a36-4697-ab13-262697a88309 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:41.475107 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fa294-8a36-4697-ab13-262697a88309 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:41.475374 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fa294-8a36-4697-ab13-262697a88309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:41.479274 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9e16309a-b2ce-4628-8599-da87c49c5332 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '578ea63d03fc48c881899b6482d954e0', 'request_id': 'req-9e16309a-b2ce-4628-8599-da87c49c5332'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:41.536896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9e16309a-b2ce-4628-8599-da87c49c5332 admin admin] The token cache is being invalidate because project 578ea63d03fc48c881899b6482d954e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:41:41.537349 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9e16309a-b2ce-4628-8599-da87c49c5332 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:41.538131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9e16309a-b2ce-4628-8599-da87c49c5332 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:41.549330 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 837/1659] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:41:41 2022] DELETE /identity/v3/projects/578ea63d03fc48c881899b6482d954e0 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:41:41.551153 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b15e42-8b81-4103-b845-d70d547d599d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:41.552079 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b15e42-8b81-4103-b845-d70d547d599d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:41.552487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b15e42-8b81-4103-b845-d70d547d599d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:41.613346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-12b15e42-8b81-4103-b845-d70d547d599d None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:41.617058 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-12b15e42-8b81-4103-b845-d70d547d599d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:41.676564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-989fa294-8a36-4697-ab13-262697a88309 None None] MFA Rules not processed for user `c0ada1accf1742b393e8a12c60074b9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:41.684863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-989fa294-8a36-4697-ab13-262697a88309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:41.880675 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:41.989651 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 823/1660] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 519 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:42.006652 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989fa294-8a36-4697-ab13-262697a88309 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:42.009540 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-989fa294-8a36-4697-ab13-262697a88309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:42.061413 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 838/1661] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 511 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:42.075108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566470f-5aae-4a6d-9555-30433c93332a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:42.075486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566470f-5aae-4a6d-9555-30433c93332a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:42.075769 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566470f-5aae-4a6d-9555-30433c93332a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:42.120766 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:42.168663 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3566470f-5aae-4a6d-9555-30433c93332a None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:42.169922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3566470f-5aae-4a6d-9555-30433c93332a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:42.200791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 839/1662] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:42.215582 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566470f-5aae-4a6d-9555-30433c93332a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:42.242164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566470f-5aae-4a6d-9555-30433c93332a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:42.253076 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0384b45-0d85-49af-875a-9d10038e1ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:42.253708 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0384b45-0d85-49af-875a-9d10038e1ba0 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:42.253993 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0384b45-0d85-49af-875a-9d10038e1ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:42.254305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0384b45-0d85-49af-875a-9d10038e1ba0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:42.256510 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0384b45-0d85-49af-875a-9d10038e1ba0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:41:42.256840 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0384b45-0d85-49af-875a-9d10038e1ba0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:42.261800 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0384b45-0d85-49af-875a-9d10038e1ba0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:42.295308 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 840/1663] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:41:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:41:42.297072 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989fa294-8a36-4697-ab13-262697a88309 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:42.298918 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d4985-9b52-4dde-94a9-c7288f7cac52 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:42.299264 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d4985-9b52-4dde-94a9-c7288f7cac52 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:42.299548 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d4985-9b52-4dde-94a9-c7288f7cac52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:42.320650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c66d04-96b7-4e5b-b2e0-bd20da03416d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:42.321281 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c66d04-96b7-4e5b-b2e0-bd20da03416d service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:42.321442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c66d04-96b7-4e5b-b2e0-bd20da03416d service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:42.321797 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c66d04-96b7-4e5b-b2e0-bd20da03416d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:42.357685 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c66d04-96b7-4e5b-b2e0-bd20da03416d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:42.359754 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c66d04-96b7-4e5b-b2e0-bd20da03416d service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:42.378524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ba2d4985-9b52-4dde-94a9-c7288f7cac52 None None] MFA Rules not processed for user `570c2a77b862411083b9444f8292192b`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:42.380996 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ba2d4985-9b52-4dde-94a9-c7288f7cac52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:42.409758 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 824/1664] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:42.418977 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c66d04-96b7-4e5b-b2e0-bd20da03416d service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:42.471014 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c66d04-96b7-4e5b-b2e0-bd20da03416d service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:42.485897 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3c85d-5dba-4aef-af2f-00fd1b83943c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:42.488654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3c85d-5dba-4aef-af2f-00fd1b83943c admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:42.488654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3c85d-5dba-4aef-af2f-00fd1b83943c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:42.488654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3c85d-5dba-4aef-af2f-00fd1b83943c admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:42.488654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d3c85d-5dba-4aef-af2f-00fd1b83943c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:42.491061 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d3c85d-5dba-4aef-af2f-00fd1b83943c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:42.619656 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 841/1665] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:42.622909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d4985-9b52-4dde-94a9-c7288f7cac52 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:42.627967 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ba2d4985-9b52-4dde-94a9-c7288f7cac52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:42.670288 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d4985-9b52-4dde-94a9-c7288f7cac52 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:42.681172 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 825/1666] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:41:42 2022] POST /identity/v3/projects => generated 373 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:42.681945 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3c85d-5dba-4aef-af2f-00fd1b83943c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:42.828523 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d056d6-a8af-4ce8-a612-b28967f78231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:42.829320 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d056d6-a8af-4ce8-a612-b28967f78231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:42.829861 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d056d6-a8af-4ce8-a612-b28967f78231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:42.830316 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d056d6-a8af-4ce8-a612-b28967f78231 admin admin] PATH_INFO: `/v3/users/09c9a6a6f39c4b5fadd26ba4c53652ea` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:42.832029 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d056d6-a8af-4ce8-a612-b28967f78231 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09c9a6a6f39c4b5fadd26ba4c53652ea)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:42.833684 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: WARNING py.warnings [None req-28d056d6-a8af-4ce8-a612-b28967f78231 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:41:42.833684 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: warnings.warn(msg) Aug 10 12:41:42.833684 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: Aug 10 12:41:42.836298 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d056d6-a8af-4ce8-a612-b28967f78231 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:42.946102 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3c85d-5dba-4aef-af2f-00fd1b83943c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:42.967206 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3525b-4bd1-4851-850b-2c5f3b208477 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:42.968059 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3525b-4bd1-4851-850b-2c5f3b208477 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:42.968491 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3525b-4bd1-4851-850b-2c5f3b208477 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:42.968931 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3525b-4bd1-4851-850b-2c5f3b208477 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:43.117674 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3525b-4bd1-4851-850b-2c5f3b208477 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:43.119719 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3525b-4bd1-4851-850b-2c5f3b208477 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:43.135319 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 826/1667] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:43.138241 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3525b-4bd1-4851-850b-2c5f3b208477 service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:43.179460 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-28d056d6-a8af-4ce8-a612-b28967f78231 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09c9a6a6f39c4b5fadd26ba4c53652ea'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:43.199285 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3525b-4bd1-4851-850b-2c5f3b208477 service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:43.224905 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-28d056d6-a8af-4ce8-a612-b28967f78231 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09c9a6a6f39c4b5fadd26ba4c53652ea'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:43.225567 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b0800-5c79-4884-97fc-bab5ae4fbee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:43.228702 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b0800-5c79-4884-97fc-bab5ae4fbee5 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:43.229762 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b0800-5c79-4884-97fc-bab5ae4fbee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:43.232029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b0800-5c79-4884-97fc-bab5ae4fbee5 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:43.235805 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0b0800-5c79-4884-97fc-bab5ae4fbee5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:43.244173 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0b0800-5c79-4884-97fc-bab5ae4fbee5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:43.322057 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-28d056d6-a8af-4ce8-a612-b28967f78231 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09c9a6a6f39c4b5fadd26ba4c53652ea'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:43.322751 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-28d056d6-a8af-4ce8-a612-b28967f78231 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09c9a6a6f39c4b5fadd26ba4c53652ea'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:43.323424 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-28d056d6-a8af-4ce8-a612-b28967f78231 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09c9a6a6f39c4b5fadd26ba4c53652ea'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:43.380487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 842/1668] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:41:42 2022] DELETE /identity/v3/users/09c9a6a6f39c4b5fadd26ba4c53652ea => generated 0 bytes in 756 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:41:43.380487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d056d6-a8af-4ce8-a612-b28967f78231 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:43.383815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:43.385609 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-28d056d6-a8af-4ce8-a612-b28967f78231 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:43.392600 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 827/1669] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:41:43 2022] POST /identity/v3/users => generated 287 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:43.393960 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ecb9a-3ef3-440a-9df7-1f9ab7a54fba None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:43.394246 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ecb9a-3ef3-440a-9df7-1f9ab7a54fba None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:43.394520 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ecb9a-3ef3-440a-9df7-1f9ab7a54fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:43.473836 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f44ecb9a-3ef3-440a-9df7-1f9ab7a54fba None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:43.475108 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f44ecb9a-3ef3-440a-9df7-1f9ab7a54fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:43.590427 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d056d6-a8af-4ce8-a612-b28967f78231 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:43.606258 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b32c91-5a99-460e-b237-a47b5f06673a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:43.606924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b32c91-5a99-460e-b237-a47b5f06673a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:43.607226 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b32c91-5a99-460e-b237-a47b5f06673a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:43.607511 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b32c91-5a99-460e-b237-a47b5f06673a admin admin] PATH_INFO: `/v3/users/0dfdede4a0184e1d9830a40adec0226b` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:43.608933 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b32c91-5a99-460e-b237-a47b5f06673a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dfdede4a0184e1d9830a40adec0226b)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:43.611275 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b32c91-5a99-460e-b237-a47b5f06673a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:43.644004 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 828/1670] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:43.645712 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ecb9a-3ef3-440a-9df7-1f9ab7a54fba None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:43.655977 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f44ecb9a-3ef3-440a-9df7-1f9ab7a54fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:43.678419 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ecb9a-3ef3-440a-9df7-1f9ab7a54fba None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:43.696610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566065d7-e83a-4609-9cfd-26b362d11bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:43.697250 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566065d7-e83a-4609-9cfd-26b362d11bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:43.697561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566065d7-e83a-4609-9cfd-26b362d11bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:43.697837 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566065d7-e83a-4609-9cfd-26b362d11bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:43.698495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566065d7-e83a-4609-9cfd-26b362d11bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:43.700165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566065d7-e83a-4609-9cfd-26b362d11bc8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:43.722824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 829/1671] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:43.727732 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566065d7-e83a-4609-9cfd-26b362d11bc8 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:43.749257 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566065d7-e83a-4609-9cfd-26b362d11bc8 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:43.770941 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42cc4f4-1b51-4db8-b439-75d236357938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:43.774818 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cc4f4-1b51-4db8-b439-75d236357938 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:43.774818 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cc4f4-1b51-4db8-b439-75d236357938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:43.774818 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cc4f4-1b51-4db8-b439-75d236357938 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:43.774818 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42cc4f4-1b51-4db8-b439-75d236357938 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:43.774818 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42cc4f4-1b51-4db8-b439-75d236357938 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:43.793746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 830/1672] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:41:43.804611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42cc4f4-1b51-4db8-b439-75d236357938 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:43.835674 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42cc4f4-1b51-4db8-b439-75d236357938 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:43.856159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05f54e-09fd-4bc2-84ae-2a83e18d8b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:43.856841 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05f54e-09fd-4bc2-84ae-2a83e18d8b73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:43.858704 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05f54e-09fd-4bc2-84ae-2a83e18d8b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:43.858996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05f54e-09fd-4bc2-84ae-2a83e18d8b73 admin admin] PATH_INFO: `/v3/projects/00fa5ee6d2034dde9813e4d0a141aec4/users/7f0f57daed0843abbc46a7903ce0b190/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:43.891767 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d7b32c91-5a99-460e-b237-a47b5f06673a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dfdede4a0184e1d9830a40adec0226b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:43.913186 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d7b32c91-5a99-460e-b237-a47b5f06673a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dfdede4a0184e1d9830a40adec0226b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:43.915703 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05f54e-09fd-4bc2-84ae-2a83e18d8b73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00fa5ee6d2034dde9813e4d0a141aec4, user_id=7f0f57daed0843abbc46a7903ce0b190, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:43.918486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05f54e-09fd-4bc2-84ae-2a83e18d8b73 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:44.060976 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:44.063606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d7b32c91-5a99-460e-b237-a47b5f06673a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dfdede4a0184e1d9830a40adec0226b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:44.064512 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d7b32c91-5a99-460e-b237-a47b5f06673a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dfdede4a0184e1d9830a40adec0226b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:44.065613 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d7b32c91-5a99-460e-b237-a47b5f06673a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dfdede4a0184e1d9830a40adec0226b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:44.068994 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 831/1673] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:41:43 2022] PUT /identity/v3/projects/00fa5ee6d2034dde9813e4d0a141aec4/users/7f0f57daed0843abbc46a7903ce0b190/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:41:44.070888 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05f54e-09fd-4bc2-84ae-2a83e18d8b73 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:44.075802 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4d05f54e-09fd-4bc2-84ae-2a83e18d8b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:44.118181 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 843/1674] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:41:43 2022] DELETE /identity/v3/users/0dfdede4a0184e1d9830a40adec0226b => generated 0 bytes in 737 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:41:44.123355 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf147f9-6c25-4ce5-b96e-c3837d4faa1f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:44.124124 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf147f9-6c25-4ce5-b96e-c3837d4faa1f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:44.124407 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf147f9-6c25-4ce5-b96e-c3837d4faa1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:44.238544 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-daf147f9-6c25-4ce5-b96e-c3837d4faa1f None None] MFA Rules not processed for user `7f0f57daed0843abbc46a7903ce0b190`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:44.241192 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-daf147f9-6c25-4ce5-b96e-c3837d4faa1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:44.282215 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05f54e-09fd-4bc2-84ae-2a83e18d8b73 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:44.476034 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99c753-c5f5-430b-966b-a3d6f0fea6d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:44.478948 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99c753-c5f5-430b-966b-a3d6f0fea6d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:44.478948 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99c753-c5f5-430b-966b-a3d6f0fea6d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:44.478948 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99c753-c5f5-430b-966b-a3d6f0fea6d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:44.480103 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8c99c753-c5f5-430b-966b-a3d6f0fea6d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:44.482518 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 844/1675] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:44.483829 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e132c1-3ec7-4b6e-b937-ee66fc2575df None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:44.484115 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e132c1-3ec7-4b6e-b937-ee66fc2575df None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:44.484395 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e132c1-3ec7-4b6e-b937-ee66fc2575df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:44.534042 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-65e132c1-3ec7-4b6e-b937-ee66fc2575df None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:44.537678 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-65e132c1-3ec7-4b6e-b937-ee66fc2575df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:44.558906 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:44.655383 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99c753-c5f5-430b-966b-a3d6f0fea6d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:44.656879 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99c753-c5f5-430b-966b-a3d6f0fea6d8 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:44.669651 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 832/1676] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:44.671696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35454a4-8752-419a-addc-5dc65a0073b8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:44.672627 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35454a4-8752-419a-addc-5dc65a0073b8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:44.678069 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35454a4-8752-419a-addc-5dc65a0073b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:44.688074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 845/1677] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:44.689243 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e132c1-3ec7-4b6e-b937-ee66fc2575df None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:44.693031 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-65e132c1-3ec7-4b6e-b937-ee66fc2575df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:44.740634 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e132c1-3ec7-4b6e-b937-ee66fc2575df None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:44.749685 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4249f-ce74-4e4f-92c9-b9907af5de0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:44.750995 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4249f-ce74-4e4f-92c9-b9907af5de0f admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:44.751518 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4249f-ce74-4e4f-92c9-b9907af5de0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:44.752011 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4249f-ce74-4e4f-92c9-b9907af5de0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:44.752927 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d4249f-ce74-4e4f-92c9-b9907af5de0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:44.756003 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d4249f-ce74-4e4f-92c9-b9907af5de0f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:44.767239 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d35454a4-8752-419a-addc-5dc65a0073b8 None None] MFA Rules not processed for user `e42059e9af9842548ea08dcd39ff0b6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:44.768509 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d35454a4-8752-419a-addc-5dc65a0073b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:44.878807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 846/1678] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:41:44 2022] POST /identity/v3/projects => generated 373 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:44.881011 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4249f-ce74-4e4f-92c9-b9907af5de0f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:44.884968 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-61d4249f-ce74-4e4f-92c9-b9907af5de0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:44.897768 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 833/1679] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 10 12:41:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:44.900163 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35454a4-8752-419a-addc-5dc65a0073b8 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:44.933899 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35454a4-8752-419a-addc-5dc65a0073b8 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:45.003983 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4249f-ce74-4e4f-92c9-b9907af5de0f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:45.009841 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ae431-e557-4895-87d7-324656e67754 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:45.011020 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ae431-e557-4895-87d7-324656e67754 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:45.012139 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ae431-e557-4895-87d7-324656e67754 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:45.013308 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ae431-e557-4895-87d7-324656e67754 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:45.017047 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380b499-ced3-44e8-8f6f-72a34cae0d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:45.017047 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380b499-ced3-44e8-8f6f-72a34cae0d6c admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:45.017047 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380b499-ced3-44e8-8f6f-72a34cae0d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:45.017047 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380b499-ced3-44e8-8f6f-72a34cae0d6c admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:45.017443 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d380b499-ced3-44e8-8f6f-72a34cae0d6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:45.019299 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d380b499-ced3-44e8-8f6f-72a34cae0d6c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:45.027320 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267ae431-e557-4895-87d7-324656e67754 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:45.030801 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267ae431-e557-4895-87d7-324656e67754 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:45.046828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 847/1680] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:45.202172 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 834/1681] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:41:44 2022] POST /identity/v3/users => generated 293 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:45.206320 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ae431-e557-4895-87d7-324656e67754 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:45.223955 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ae431-e557-4895-87d7-324656e67754 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:45.240882 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed578119-d6db-40d2-9a40-53cdf368cc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:45.243757 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed578119-d6db-40d2-9a40-53cdf368cc46 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:45.244320 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed578119-d6db-40d2-9a40-53cdf368cc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:45.244604 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed578119-d6db-40d2-9a40-53cdf368cc46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:45.250913 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed578119-d6db-40d2-9a40-53cdf368cc46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:45.251077 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed578119-d6db-40d2-9a40-53cdf368cc46 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:45.279621 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 848/1682] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:45 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:45.284102 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380b499-ced3-44e8-8f6f-72a34cae0d6c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:45.307612 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380b499-ced3-44e8-8f6f-72a34cae0d6c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:45.323444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303bd9c-03c1-4c9c-a753-353965ed6dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:45.326113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303bd9c-03c1-4c9c-a753-353965ed6dad admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:45.326418 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303bd9c-03c1-4c9c-a753-353965ed6dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:45.326697 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303bd9c-03c1-4c9c-a753-353965ed6dad admin admin] PATH_INFO: `/v3/projects/81e028c37f334ce6bbcce6f6ba570c74/users/872d4715974f480199edb4a571441bc0/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:45.366609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b303bd9c-03c1-4c9c-a753-353965ed6dad admin admin] RBAC: Authorizing `identity:create_grant(project_id=81e028c37f334ce6bbcce6f6ba570c74, user_id=872d4715974f480199edb4a571441bc0, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:45.384434 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b303bd9c-03c1-4c9c-a753-353965ed6dad admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:45.420932 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed578119-d6db-40d2-9a40-53cdf368cc46 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:45.422725 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ed578119-d6db-40d2-9a40-53cdf368cc46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:45.475756 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 835/1683] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:41:45 2022] PUT /identity/v3/projects/81e028c37f334ce6bbcce6f6ba570c74/users/872d4715974f480199edb4a571441bc0/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:41:45.477009 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303bd9c-03c1-4c9c-a753-353965ed6dad admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:45.646095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed578119-d6db-40d2-9a40-53cdf368cc46 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:45.655136 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303bd9c-03c1-4c9c-a753-353965ed6dad admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:45.668577 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8835d6-a6ee-4115-b39e-22186c60ee5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:45.670291 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8835d6-a6ee-4115-b39e-22186c60ee5e admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:45.670586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8835d6-a6ee-4115-b39e-22186c60ee5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:45.670840 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8835d6-a6ee-4115-b39e-22186c60ee5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:45.671523 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8835d6-a6ee-4115-b39e-22186c60ee5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:45.680601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8835d6-a6ee-4115-b39e-22186c60ee5e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:45.719537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 836/1684] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:45 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:45.723855 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8835d6-a6ee-4115-b39e-22186c60ee5e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:45.730922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23d448-c14d-41c7-9b9b-55561e32d7e2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59395bfa505c4540b3957deee78eb698', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:45.742223 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:45.746808 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23d448-c14d-41c7-9b9b-55561e32d7e2 service swift] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:45.753036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8835d6-a6ee-4115-b39e-22186c60ee5e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:45.756483 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23d448-c14d-41c7-9b9b-55561e32d7e2 service swift] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:45.757374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23d448-c14d-41c7-9b9b-55561e32d7e2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:45.767470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e5f277-06f7-4cd6-81d7-23f3824e7274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:45.768164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5f277-06f7-4cd6-81d7-23f3824e7274 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:45.768451 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5f277-06f7-4cd6-81d7-23f3824e7274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:45.768728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5f277-06f7-4cd6-81d7-23f3824e7274 admin admin] PATH_INFO: `/v3/projects/81e028c37f334ce6bbcce6f6ba570c74/users/872d4715974f480199edb4a571441bc0/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:45.772736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e5f277-06f7-4cd6-81d7-23f3824e7274 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81e028c37f334ce6bbcce6f6ba570c74, user_id=872d4715974f480199edb4a571441bc0, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:45.781118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e5f277-06f7-4cd6-81d7-23f3824e7274 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:45.785133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:45.809997 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 837/1685] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:41:45 2022] PUT /identity/v3/projects/81e028c37f334ce6bbcce6f6ba570c74/users/872d4715974f480199edb4a571441bc0/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:41:45.826044 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acb245-0e93-4c32-bddd-175e2b8a6aa6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:45.826044 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acb245-0e93-4c32-bddd-175e2b8a6aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:45.826044 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acb245-0e93-4c32-bddd-175e2b8a6aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:45.886856 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-23acb245-0e93-4c32-bddd-175e2b8a6aa6 None None] MFA Rules not processed for user `872d4715974f480199edb4a571441bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:45.888482 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-23acb245-0e93-4c32-bddd-175e2b8a6aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:45.915948 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b23d448-c14d-41c7-9b9b-55561e32d7e2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:45.917890 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b23d448-c14d-41c7-9b9b-55561e32d7e2 service swift] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:45.965642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 849/1686] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 12:41:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 12:41:46.206391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 838/1687] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:45 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:41:46.221163 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23d448-c14d-41c7-9b9b-55561e32d7e2 service swift] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:46.226440 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9b23d448-c14d-41c7-9b9b-55561e32d7e2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:46.417679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23d448-c14d-41c7-9b9b-55561e32d7e2 service swift] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:46.584030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b234a12-8ec9-41aa-a942-85c990b1d8f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:46.584701 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b234a12-8ec9-41aa-a942-85c990b1d8f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:46.588316 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b234a12-8ec9-41aa-a942-85c990b1d8f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:46.589540 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b234a12-8ec9-41aa-a942-85c990b1d8f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:46.638979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b234a12-8ec9-41aa-a942-85c990b1d8f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:46.641941 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b234a12-8ec9-41aa-a942-85c990b1d8f8 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:46.709761 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 850/1688] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:46.970950 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23acb245-0e93-4c32-bddd-175e2b8a6aa6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:46.972194 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b234a12-8ec9-41aa-a942-85c990b1d8f8 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:46.975959 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2b234a12-8ec9-41aa-a942-85c990b1d8f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:46.991974 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23acb245-0e93-4c32-bddd-175e2b8a6aa6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:47.004336 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c12ef-afae-48d2-805d-383300d22168 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:47.007346 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c12ef-afae-48d2-805d-383300d22168 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:47.007824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c12ef-afae-48d2-805d-383300d22168 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:47.008801 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c12ef-afae-48d2-805d-383300d22168 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:47.228017 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c12ef-afae-48d2-805d-383300d22168 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:47.229489 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c12ef-afae-48d2-805d-383300d22168 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:47.263525 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 839/1689] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:47.344145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b234a12-8ec9-41aa-a942-85c990b1d8f8 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:47.377724 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c821f8-50c3-4131-afd9-156575df9960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:47.378399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c821f8-50c3-4131-afd9-156575df9960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:47.388451 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c821f8-50c3-4131-afd9-156575df9960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:47.388756 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c821f8-50c3-4131-afd9-156575df9960 admin admin] PATH_INFO: `/v3/projects/e9cc496fab1444c8b5ea5d98e48d3304` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:47.392666 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c821f8-50c3-4131-afd9-156575df9960 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9cc496fab1444c8b5ea5d98e48d3304)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:47.394719 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c821f8-50c3-4131-afd9-156575df9960 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:47.425680 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:47.684300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a5c821f8-50c3-4131-afd9-156575df9960 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9cc496fab1444c8b5ea5d98e48d3304', 'request_id': 'req-a5c821f8-50c3-4131-afd9-156575df9960'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:47.685274 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a5c821f8-50c3-4131-afd9-156575df9960 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9cc496fab1444c8b5ea5d98e48d3304', 'request_id': 'req-a5c821f8-50c3-4131-afd9-156575df9960'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:47.917049 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a5c821f8-50c3-4131-afd9-156575df9960 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9cc496fab1444c8b5ea5d98e48d3304', 'request_id': 'req-a5c821f8-50c3-4131-afd9-156575df9960'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:47.949043 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c12ef-afae-48d2-805d-383300d22168 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:47.950798 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b32c12ef-afae-48d2-805d-383300d22168 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:47.967765 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a5c821f8-50c3-4131-afd9-156575df9960 admin admin] The token cache is being invalidate because project e9cc496fab1444c8b5ea5d98e48d3304 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:41:47.968126 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a5c821f8-50c3-4131-afd9-156575df9960 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:47.972112 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a5c821f8-50c3-4131-afd9-156575df9960 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:47.975845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:47.977424 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 851/1690] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:41:46 2022] DELETE /identity/v3/projects/e9cc496fab1444c8b5ea5d98e48d3304 => generated 0 bytes in 1008 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:41:47.982179 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c821f8-50c3-4131-afd9-156575df9960 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:47.988529 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a5c821f8-50c3-4131-afd9-156575df9960 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:47.998604 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c12ef-afae-48d2-805d-383300d22168 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:48.241505 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c821f8-50c3-4131-afd9-156575df9960 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:48.260793 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1a680-6072-4e50-815e-d2a8545989ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:48.264685 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1a680-6072-4e50-815e-d2a8545989ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:48.264685 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1a680-6072-4e50-815e-d2a8545989ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:48.264685 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1a680-6072-4e50-815e-d2a8545989ac admin admin] PATH_INFO: `/v3/projects/54fe7893428b4bb09e7b3642e04c614f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:48.264685 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1a680-6072-4e50-815e-d2a8545989ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=54fe7893428b4bb09e7b3642e04c614f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:48.267580 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6499d1a-31a4-4ae8-a580-7951160aa1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:48.268230 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6499d1a-31a4-4ae8-a580-7951160aa1b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:48.268581 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6499d1a-31a4-4ae8-a580-7951160aa1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:48.268975 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6499d1a-31a4-4ae8-a580-7951160aa1b4 admin admin] PATH_INFO: `/v3/users/a1af427b45c54ed1b1166e154a103891` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:48.270780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6499d1a-31a4-4ae8-a580-7951160aa1b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1af427b45c54ed1b1166e154a103891)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:48.273473 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6499d1a-31a4-4ae8-a580-7951160aa1b4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:48.287911 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: WARNING py.warnings [None req-52f1a680-6072-4e50-815e-d2a8545989ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:41:48.287911 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: warnings.warn(msg) Aug 10 12:41:48.287911 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: Aug 10 12:41:48.289166 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1a680-6072-4e50-815e-d2a8545989ac admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:48.479424 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:48.524727 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-52f1a680-6072-4e50-815e-d2a8545989ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54fe7893428b4bb09e7b3642e04c614f', 'request_id': 'req-52f1a680-6072-4e50-815e-d2a8545989ac'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:48.533273 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-52f1a680-6072-4e50-815e-d2a8545989ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54fe7893428b4bb09e7b3642e04c614f', 'request_id': 'req-52f1a680-6072-4e50-815e-d2a8545989ac'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:48.608272 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-52f1a680-6072-4e50-815e-d2a8545989ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54fe7893428b4bb09e7b3642e04c614f', 'request_id': 'req-52f1a680-6072-4e50-815e-d2a8545989ac'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:48.638224 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b6499d1a-31a4-4ae8-a580-7951160aa1b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1af427b45c54ed1b1166e154a103891'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:48.655043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b6499d1a-31a4-4ae8-a580-7951160aa1b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1af427b45c54ed1b1166e154a103891'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:48.667228 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-52f1a680-6072-4e50-815e-d2a8545989ac admin admin] The token cache is being invalidate because project 54fe7893428b4bb09e7b3642e04c614f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:41:48.668995 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-52f1a680-6072-4e50-815e-d2a8545989ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:48.670981 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-52f1a680-6072-4e50-815e-d2a8545989ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:48.676455 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 840/1691] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:41:47 2022] DELETE /identity/v3/projects/54fe7893428b4bb09e7b3642e04c614f => generated 0 bytes in 728 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:41:48.678589 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22265137-7b0c-468f-80b4-d59dc9198000 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:48.684617 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22265137-7b0c-468f-80b4-d59dc9198000 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:48.686594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22265137-7b0c-468f-80b4-d59dc9198000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:48.738852 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-22265137-7b0c-468f-80b4-d59dc9198000 None None] MFA Rules not processed for user `5db814952f9f466abf6148b7a0db2173`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:48.744573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-22265137-7b0c-468f-80b4-d59dc9198000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:48.760020 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b6499d1a-31a4-4ae8-a580-7951160aa1b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1af427b45c54ed1b1166e154a103891'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:48.760020 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b6499d1a-31a4-4ae8-a580-7951160aa1b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1af427b45c54ed1b1166e154a103891'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:48.760020 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b6499d1a-31a4-4ae8-a580-7951160aa1b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1af427b45c54ed1b1166e154a103891'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:48.810210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 852/1692] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:41:47 2022] DELETE /identity/v3/users/a1af427b45c54ed1b1166e154a103891 => generated 0 bytes in 828 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:41:48.821675 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc52047-937b-4ee4-b302-3d3fb904ed48 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:48.824114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc52047-937b-4ee4-b302-3d3fb904ed48 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:48.824688 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc52047-937b-4ee4-b302-3d3fb904ed48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:48.916899 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 841/1693] 10.222.0.55 () {58 vars in 1015 bytes} [Wed Aug 10 12:41:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:48.931786 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6fc52047-937b-4ee4-b302-3d3fb904ed48 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:48.936876 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6fc52047-937b-4ee4-b302-3d3fb904ed48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:49.029354 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:49.066656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22265137-7b0c-468f-80b4-d59dc9198000 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:49.069069 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-22265137-7b0c-468f-80b4-d59dc9198000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:49.086539 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22265137-7b0c-468f-80b4-d59dc9198000 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:49.151984 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 853/1694] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:49.157520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc52047-937b-4ee4-b302-3d3fb904ed48 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:49.184139 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:49.217807 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1aabe2-e6aa-433e-8f95-7171be2fce67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:49.220805 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1aabe2-e6aa-433e-8f95-7171be2fce67 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:49.220805 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1aabe2-e6aa-433e-8f95-7171be2fce67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:49.220805 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1aabe2-e6aa-433e-8f95-7171be2fce67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:49.231698 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1aabe2-e6aa-433e-8f95-7171be2fce67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:49.235362 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1aabe2-e6aa-433e-8f95-7171be2fce67 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:49.259558 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 842/1695] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:49.261019 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960d8a4-2da6-46f1-8d12-ab7aca008361 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:49.261304 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960d8a4-2da6-46f1-8d12-ab7aca008361 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:49.261760 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960d8a4-2da6-46f1-8d12-ab7aca008361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:49.339789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a960d8a4-2da6-46f1-8d12-ab7aca008361 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:49.339789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a960d8a4-2da6-46f1-8d12-ab7aca008361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:49.354183 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 843/1696] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:49.362137 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960d8a4-2da6-46f1-8d12-ab7aca008361 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:49.374075 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc52047-937b-4ee4-b302-3d3fb904ed48 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:49.387882 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08dd4a8-a59d-4446-bf0c-fa0e4d1e86f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:49.388840 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dd4a8-a59d-4446-bf0c-fa0e4d1e86f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:49.389320 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dd4a8-a59d-4446-bf0c-fa0e4d1e86f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:49.389790 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dd4a8-a59d-4446-bf0c-fa0e4d1e86f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:49.391253 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960d8a4-2da6-46f1-8d12-ab7aca008361 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:49.403593 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61925ab4-52f3-4a38-942e-29e6d35ea577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:49.404488 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61925ab4-52f3-4a38-942e-29e6d35ea577 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:49.404914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61925ab4-52f3-4a38-942e-29e6d35ea577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:49.407840 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61925ab4-52f3-4a38-942e-29e6d35ea577 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:49.408692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61925ab4-52f3-4a38-942e-29e6d35ea577 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:41:49.409173 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61925ab4-52f3-4a38-942e-29e6d35ea577 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:49.417648 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61925ab4-52f3-4a38-942e-29e6d35ea577 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:49.428375 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08dd4a8-a59d-4446-bf0c-fa0e4d1e86f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:49.429897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08dd4a8-a59d-4446-bf0c-fa0e4d1e86f3 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:49.454555 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 854/1697] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:49.466296 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 844/1698] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:41:49.478597 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08dd4a8-a59d-4446-bf0c-fa0e4d1e86f3 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:49.523416 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08dd4a8-a59d-4446-bf0c-fa0e4d1e86f3 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:49.529230 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d86985-1994-4532-84ba-ab02765c2b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:49.530158 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d86985-1994-4532-84ba-ab02765c2b2a admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:49.530786 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d86985-1994-4532-84ba-ab02765c2b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:49.531069 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d86985-1994-4532-84ba-ab02765c2b2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:49.531900 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d86985-1994-4532-84ba-ab02765c2b2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:49.534139 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d86985-1994-4532-84ba-ab02765c2b2a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:49.652527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 855/1699] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:41:49 2022] POST /identity/v3/projects => generated 387 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:49.653739 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61925ab4-52f3-4a38-942e-29e6d35ea577 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:49.670318 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61925ab4-52f3-4a38-942e-29e6d35ea577 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:49.820336 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d423e384-cd68-4220-a460-73c6e6de25cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:49.825917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423e384-cd68-4220-a460-73c6e6de25cf admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:49.827801 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423e384-cd68-4220-a460-73c6e6de25cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:49.828985 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423e384-cd68-4220-a460-73c6e6de25cf admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:49.830353 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d423e384-cd68-4220-a460-73c6e6de25cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:49.840140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d423e384-cd68-4220-a460-73c6e6de25cf admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:50.024506 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 845/1700] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:41:49 2022] POST /identity/v3/users => generated 294 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:50.024720 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d86985-1994-4532-84ba-ab02765c2b2a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:50.044285 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d86985-1994-4532-84ba-ab02765c2b2a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:50.062502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951c7d8-a3e3-4193-b630-9cbdf29ea0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:50.065719 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951c7d8-a3e3-4193-b630-9cbdf29ea0ef admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:50.070031 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951c7d8-a3e3-4193-b630-9cbdf29ea0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:50.070673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951c7d8-a3e3-4193-b630-9cbdf29ea0ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:50.072520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951c7d8-a3e3-4193-b630-9cbdf29ea0ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:50.085342 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951c7d8-a3e3-4193-b630-9cbdf29ea0ef admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:50.116043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 856/1701] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:50.118377 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d423e384-cd68-4220-a460-73c6e6de25cf admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:50.134422 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d423e384-cd68-4220-a460-73c6e6de25cf admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:50.152835 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1412f1-247b-4941-a2db-1b11b3bf4e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:50.153668 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1412f1-247b-4941-a2db-1b11b3bf4e99 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:50.154464 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1412f1-247b-4941-a2db-1b11b3bf4e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:50.154942 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1412f1-247b-4941-a2db-1b11b3bf4e99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:50.158589 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1412f1-247b-4941-a2db-1b11b3bf4e99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:50.162006 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1412f1-247b-4941-a2db-1b11b3bf4e99 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:50.204801 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 846/1702] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:50.205742 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951c7d8-a3e3-4193-b630-9cbdf29ea0ef admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:50.216570 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:50.227070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951c7d8-a3e3-4193-b630-9cbdf29ea0ef admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:50.245730 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc294adf-6a30-464a-a744-7db2ecbb7852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:50.246609 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc294adf-6a30-464a-a744-7db2ecbb7852 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:50.247043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc294adf-6a30-464a-a744-7db2ecbb7852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:50.247479 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc294adf-6a30-464a-a744-7db2ecbb7852 admin admin] PATH_INFO: `/v3/projects/cbb57fe80e9e404c923543980c601ccb/users/7f0b370d2b55410e85143d10dd1d7294/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:50.272764 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc294adf-6a30-464a-a744-7db2ecbb7852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbb57fe80e9e404c923543980c601ccb, user_id=7f0b370d2b55410e85143d10dd1d7294, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:50.276584 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc294adf-6a30-464a-a744-7db2ecbb7852 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:50.404391 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 857/1703] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:41:50 2022] PUT /identity/v3/projects/cbb57fe80e9e404c923543980c601ccb/users/7f0b370d2b55410e85143d10dd1d7294/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:41:50.418000 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02b87d-ac36-4042-9d93-d1001017148d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:50.418482 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02b87d-ac36-4042-9d93-d1001017148d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:50.418980 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02b87d-ac36-4042-9d93-d1001017148d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:50.486149 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507b541-cb11-4546-9e9e-f00f694f1f80 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:50.486820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507b541-cb11-4546-9e9e-f00f694f1f80 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:50.487351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507b541-cb11-4546-9e9e-f00f694f1f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:50.538274 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3c02b87d-ac36-4042-9d93-d1001017148d None None] MFA Rules not processed for user `7f0b370d2b55410e85143d10dd1d7294`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:50.539559 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3c02b87d-ac36-4042-9d93-d1001017148d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:50.547429 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b507b541-cb11-4546-9e9e-f00f694f1f80 None None] MFA Rules not processed for user `70010d27b7f44bc699ea5ad8093fec3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:50.564377 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b507b541-cb11-4546-9e9e-f00f694f1f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:50.756633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 858/1704] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:41:50.773761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 847/1705] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:50.778017 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507b541-cb11-4546-9e9e-f00f694f1f80 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:50.969625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507b541-cb11-4546-9e9e-f00f694f1f80 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:50.978746 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a08d1d-e669-4dff-ab8c-a6855b06e590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:50.980118 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a08d1d-e669-4dff-ab8c-a6855b06e590 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:50.980576 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a08d1d-e669-4dff-ab8c-a6855b06e590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:50.981132 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a08d1d-e669-4dff-ab8c-a6855b06e590 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:50.985408 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a08d1d-e669-4dff-ab8c-a6855b06e590 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:50.988473 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a08d1d-e669-4dff-ab8c-a6855b06e590 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:51.116890 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 859/1706] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:41:50 2022] POST /identity/v3/projects => generated 387 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:51.121541 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02b87d-ac36-4042-9d93-d1001017148d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:51.153487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02b87d-ac36-4042-9d93-d1001017148d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:51.254811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:51.313701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b549f7b-34db-45d9-8a2a-c3f0d8d290fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:51.314248 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b549f7b-34db-45d9-8a2a-c3f0d8d290fd admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:51.314530 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b549f7b-34db-45d9-8a2a-c3f0d8d290fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:51.314798 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b549f7b-34db-45d9-8a2a-c3f0d8d290fd admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:51.315453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b549f7b-34db-45d9-8a2a-c3f0d8d290fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:51.317496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b549f7b-34db-45d9-8a2a-c3f0d8d290fd admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:51.447677 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 848/1707] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:41:51 2022] POST /identity/v3/users => generated 300 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:51.448101 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a08d1d-e669-4dff-ab8c-a6855b06e590 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:51.467037 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a08d1d-e669-4dff-ab8c-a6855b06e590 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:51.492755 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4237ed-2fe9-40fe-b4dd-91d0f24ea995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:51.493702 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4237ed-2fe9-40fe-b4dd-91d0f24ea995 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:51.494366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4237ed-2fe9-40fe-b4dd-91d0f24ea995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:51.494817 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4237ed-2fe9-40fe-b4dd-91d0f24ea995 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:51.495706 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4237ed-2fe9-40fe-b4dd-91d0f24ea995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:51.501333 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4237ed-2fe9-40fe-b4dd-91d0f24ea995 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:51.529554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b549f7b-34db-45d9-8a2a-c3f0d8d290fd admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:51.538745 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 860/1708] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:41:51.548642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b549f7b-34db-45d9-8a2a-c3f0d8d290fd admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:51.556103 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a2fb0-3d80-437a-b653-6dab149f2439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:51.557035 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a2fb0-3d80-437a-b653-6dab149f2439 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:51.558148 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a2fb0-3d80-437a-b653-6dab149f2439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:51.558148 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a2fb0-3d80-437a-b653-6dab149f2439 admin admin] PATH_INFO: `/v3/projects/97b2bc810e014edf9b3e14557a34eb7e/users/06104c78dade47f7be89e4347390ee64/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:51.614889 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743a2fb0-3d80-437a-b653-6dab149f2439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97b2bc810e014edf9b3e14557a34eb7e, user_id=06104c78dade47f7be89e4347390ee64, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:51.623310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743a2fb0-3d80-437a-b653-6dab149f2439 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:51.694871 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4237ed-2fe9-40fe-b4dd-91d0f24ea995 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:51.696709 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6a4237ed-2fe9-40fe-b4dd-91d0f24ea995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:51.730447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 849/1709] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:41:51 2022] PUT /identity/v3/projects/97b2bc810e014edf9b3e14557a34eb7e/users/06104c78dade47f7be89e4347390ee64/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:41:51.734346 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a2fb0-3d80-437a-b653-6dab149f2439 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:51.750703 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4237ed-2fe9-40fe-b4dd-91d0f24ea995 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:51.928492 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a2fb0-3d80-437a-b653-6dab149f2439 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:51.943080 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ad092-8fa8-4def-9a40-5cd0060b60c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:51.945006 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ad092-8fa8-4def-9a40-5cd0060b60c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:51.945881 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ad092-8fa8-4def-9a40-5cd0060b60c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:51.946545 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ad092-8fa8-4def-9a40-5cd0060b60c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:51.949737 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ad092-8fa8-4def-9a40-5cd0060b60c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:51.956591 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ad092-8fa8-4def-9a40-5cd0060b60c9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:51.997104 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 850/1710] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:52.000541 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c5c11c-5163-419c-a7bb-953edc2a2ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:52.001878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5c11c-5163-419c-a7bb-953edc2a2ac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:52.005363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5c11c-5163-419c-a7bb-953edc2a2ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:52.005510 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ad092-8fa8-4def-9a40-5cd0060b60c9 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:52.005930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5c11c-5163-419c-a7bb-953edc2a2ac3 admin admin] PATH_INFO: `/v3/projects/239a79a4e777461f826f2a66c544a964` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:52.008144 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c5c11c-5163-419c-a7bb-953edc2a2ac3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=239a79a4e777461f826f2a66c544a964)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:52.012749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c5c11c-5163-419c-a7bb-953edc2a2ac3 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:52.042732 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ad092-8fa8-4def-9a40-5cd0060b60c9 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:52.055904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77019a4-67db-4940-8c72-cd606d3255ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:52.057409 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77019a4-67db-4940-8c72-cd606d3255ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:52.057730 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77019a4-67db-4940-8c72-cd606d3255ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:52.058071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77019a4-67db-4940-8c72-cd606d3255ec admin admin] PATH_INFO: `/v3/projects/97b2bc810e014edf9b3e14557a34eb7e/users/06104c78dade47f7be89e4347390ee64/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:52.071857 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77019a4-67db-4940-8c72-cd606d3255ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=97b2bc810e014edf9b3e14557a34eb7e, user_id=06104c78dade47f7be89e4347390ee64, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:52.080300 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77019a4-67db-4940-8c72-cd606d3255ec admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:52.116613 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 851/1711] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:41:51 2022] PUT /identity/v3/projects/97b2bc810e014edf9b3e14557a34eb7e/users/06104c78dade47f7be89e4347390ee64/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:41:52.126464 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0b992-9f3e-4607-898a-579b5490b234 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:52.127137 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0b992-9f3e-4607-898a-579b5490b234 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:52.127137 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0b992-9f3e-4607-898a-579b5490b234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:52.223757 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-26c5c11c-5163-419c-a7bb-953edc2a2ac3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '239a79a4e777461f826f2a66c544a964', 'request_id': 'req-26c5c11c-5163-419c-a7bb-953edc2a2ac3'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:52.227476 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-88e0b992-9f3e-4607-898a-579b5490b234 None None] MFA Rules not processed for user `06104c78dade47f7be89e4347390ee64`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:52.227476 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-88e0b992-9f3e-4607-898a-579b5490b234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:52.241884 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-26c5c11c-5163-419c-a7bb-953edc2a2ac3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '239a79a4e777461f826f2a66c544a964', 'request_id': 'req-26c5c11c-5163-419c-a7bb-953edc2a2ac3'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:52.345994 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-26c5c11c-5163-419c-a7bb-953edc2a2ac3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '239a79a4e777461f826f2a66c544a964', 'request_id': 'req-26c5c11c-5163-419c-a7bb-953edc2a2ac3'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:52.387468 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-26c5c11c-5163-419c-a7bb-953edc2a2ac3 admin admin] The token cache is being invalidate because project 239a79a4e777461f826f2a66c544a964 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:41:52.387468 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-26c5c11c-5163-419c-a7bb-953edc2a2ac3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:52.387468 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-26c5c11c-5163-419c-a7bb-953edc2a2ac3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:52.392082 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 861/1712] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:41:51 2022] DELETE /identity/v3/projects/239a79a4e777461f826f2a66c544a964 => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:41:52.394492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c281cd0-1ecd-4d08-a08c-0c23e164b30d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:52.395443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c281cd0-1ecd-4d08-a08c-0c23e164b30d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:52.395895 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c281cd0-1ecd-4d08-a08c-0c23e164b30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:52.418219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 852/1713] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:52 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:41:52.426749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-1c281cd0-1ecd-4d08-a08c-0c23e164b30d None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:52.428261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1c281cd0-1ecd-4d08-a08c-0c23e164b30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:52.432230 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e0b992-9f3e-4607-898a-579b5490b234 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:52.433702 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-88e0b992-9f3e-4607-898a-579b5490b234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:52.575394 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 862/1714] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:52.586971 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c0b45-923f-41d3-9029-1d3b1b5a53c8 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:52.588854 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c0b45-923f-41d3-9029-1d3b1b5a53c8 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:52.588854 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c0b45-923f-41d3-9029-1d3b1b5a53c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:52.606753 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e0b992-9f3e-4607-898a-579b5490b234 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:52.623901 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2fceb-78c6-4a54-b28d-7a8c597630a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:52.631537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2fceb-78c6-4a54-b28d-7a8c597630a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:52.631537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2fceb-78c6-4a54-b28d-7a8c597630a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:52.631537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2fceb-78c6-4a54-b28d-7a8c597630a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:52.631824 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-302c0b45-923f-41d3-9029-1d3b1b5a53c8 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:52.631955 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-96d2fceb-78c6-4a54-b28d-7a8c597630a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:52.641739 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-302c0b45-923f-41d3-9029-1d3b1b5a53c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:52.650471 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:52.664263 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2fceb-78c6-4a54-b28d-7a8c597630a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:52.667611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2fceb-78c6-4a54-b28d-7a8c597630a6 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:52.669305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 863/1715] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:52.675471 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c0b45-923f-41d3-9029-1d3b1b5a53c8 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:52.694731 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 853/1716] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:52 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:41:52.698715 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c0b45-923f-41d3-9029-1d3b1b5a53c8 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:52.706072 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c30613-7a06-4766-bb1f-17da7d5b9fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:52.709050 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c30613-7a06-4766-bb1f-17da7d5b9fcc admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:52.709620 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c30613-7a06-4766-bb1f-17da7d5b9fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:52.710227 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c30613-7a06-4766-bb1f-17da7d5b9fcc admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:52.711723 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c30613-7a06-4766-bb1f-17da7d5b9fcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:41:52.713088 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c30613-7a06-4766-bb1f-17da7d5b9fcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:52.715946 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c30613-7a06-4766-bb1f-17da7d5b9fcc admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:52.727321 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d11c44-006f-44e0-88a0-d64a4b6aa9af None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:52.728452 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d11c44-006f-44e0-88a0-d64a4b6aa9af None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:52.728452 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d11c44-006f-44e0-88a0-d64a4b6aa9af None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:52.729239 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 854/1717] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 10 12:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:41:52.743993 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 864/1718] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:41:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:41:52.748260 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d11c44-006f-44e0-88a0-d64a4b6aa9af None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:52.772904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d11c44-006f-44e0-88a0-d64a4b6aa9af None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:52.781215 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fa744a-0100-42a6-883b-8e5a2e66a8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:52.782174 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa744a-0100-42a6-883b-8e5a2e66a8fa admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:52.782702 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa744a-0100-42a6-883b-8e5a2e66a8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:52.783026 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa744a-0100-42a6-883b-8e5a2e66a8fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:52.783727 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fa744a-0100-42a6-883b-8e5a2e66a8fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:52.786647 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fa744a-0100-42a6-883b-8e5a2e66a8fa admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:52.828619 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c30613-7a06-4766-bb1f-17da7d5b9fcc admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:52.832250 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-31c30613-7a06-4766-bb1f-17da7d5b9fcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:52.881632 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c30613-7a06-4766-bb1f-17da7d5b9fcc admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:52.886753 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d904b433-e988-4e03-a3fb-375fc7f1ce79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:52.887599 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904b433-e988-4e03-a3fb-375fc7f1ce79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:52.893529 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904b433-e988-4e03-a3fb-375fc7f1ce79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:52.893529 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904b433-e988-4e03-a3fb-375fc7f1ce79 admin admin] PATH_INFO: `/v3/users/7f0b370d2b55410e85143d10dd1d7294` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:52.893529 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d904b433-e988-4e03-a3fb-375fc7f1ce79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f0b370d2b55410e85143d10dd1d7294)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:52.893529 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d904b433-e988-4e03-a3fb-375fc7f1ce79 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:52.985991 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 855/1719] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:41:52 2022] POST /identity/v3/projects => generated 379 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:52.987442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1576dd9-5389-4077-b117-b29275af2218 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:52.987765 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1576dd9-5389-4077-b117-b29275af2218 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:52.988216 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1576dd9-5389-4077-b117-b29275af2218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:53.023528 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e1576dd9-5389-4077-b117-b29275af2218 None None] MFA Rules not processed for user `eb3f6deffb54467f8b923956ed2a6acf`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:53.025371 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e1576dd9-5389-4077-b117-b29275af2218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:53.130190 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d904b433-e988-4e03-a3fb-375fc7f1ce79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f0b370d2b55410e85143d10dd1d7294'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:53.145556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d904b433-e988-4e03-a3fb-375fc7f1ce79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f0b370d2b55410e85143d10dd1d7294'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:53.154807 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 856/1720] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 10 12:41:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:53.155991 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1576dd9-5389-4077-b117-b29275af2218 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:53.247224 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1576dd9-5389-4077-b117-b29275af2218 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:53.253686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b47c3-060e-4b46-811c-032111a4093a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:53.255968 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b47c3-060e-4b46-811c-032111a4093a admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:53.265014 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b47c3-060e-4b46-811c-032111a4093a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:53.266325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b47c3-060e-4b46-811c-032111a4093a admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:53.266942 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3b47c3-060e-4b46-811c-032111a4093a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:53.269383 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3b47c3-060e-4b46-811c-032111a4093a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:53.271462 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d904b433-e988-4e03-a3fb-375fc7f1ce79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f0b370d2b55410e85143d10dd1d7294'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:53.272117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d904b433-e988-4e03-a3fb-375fc7f1ce79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f0b370d2b55410e85143d10dd1d7294'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:53.272765 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d904b433-e988-4e03-a3fb-375fc7f1ce79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f0b370d2b55410e85143d10dd1d7294'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:53.312539 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 865/1721] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:41:52 2022] DELETE /identity/v3/users/7f0b370d2b55410e85143d10dd1d7294 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:41:53.312539 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732dc054-3686-449e-9ca3-a61099b0ce62 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:53.312539 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732dc054-3686-449e-9ca3-a61099b0ce62 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:53.312539 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732dc054-3686-449e-9ca3-a61099b0ce62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:53.375766 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 857/1722] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:41:53 2022] POST /identity/v3/users => generated 290 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:53.378427 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-732dc054-3686-449e-9ca3-a61099b0ce62 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:53.380457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b47c3-060e-4b46-811c-032111a4093a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:53.383001 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-732dc054-3686-449e-9ca3-a61099b0ce62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:53.384555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3f3b47c3-060e-4b46-811c-032111a4093a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:53.419329 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b47c3-060e-4b46-811c-032111a4093a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:53.488364 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 866/1723] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:41:53.497810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732dc054-3686-449e-9ca3-a61099b0ce62 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:53.501181 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-732dc054-3686-449e-9ca3-a61099b0ce62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:53.513743 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:53.518101 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d102ae-e315-4c1c-8dff-f4cde35453a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:53.519144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d102ae-e315-4c1c-8dff-f4cde35453a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:53.519562 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d102ae-e315-4c1c-8dff-f4cde35453a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:53.520254 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d102ae-e315-4c1c-8dff-f4cde35453a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:53.521371 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d102ae-e315-4c1c-8dff-f4cde35453a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:53.524009 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d102ae-e315-4c1c-8dff-f4cde35453a4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:53.546248 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 858/1724] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:53 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:41:53.551018 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d102ae-e315-4c1c-8dff-f4cde35453a4 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:53.563952 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d102ae-e315-4c1c-8dff-f4cde35453a4 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:53.577944 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65305bf-d857-4283-956e-5717ee84d3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:53.579611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65305bf-d857-4283-956e-5717ee84d3d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:53.580452 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65305bf-d857-4283-956e-5717ee84d3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:53.581397 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65305bf-d857-4283-956e-5717ee84d3d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:53.583583 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65305bf-d857-4283-956e-5717ee84d3d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:53.588261 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65305bf-d857-4283-956e-5717ee84d3d0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:53.603703 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732dc054-3686-449e-9ca3-a61099b0ce62 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:53.607477 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 859/1725] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:53 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:53.609241 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba4922-466a-47a4-bb8e-a142e9457a01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:53.609871 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba4922-466a-47a4-bb8e-a142e9457a01 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:53.610435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba4922-466a-47a4-bb8e-a142e9457a01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:53.610689 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba4922-466a-47a4-bb8e-a142e9457a01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:53.612507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65305bf-d857-4283-956e-5717ee84d3d0 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:53.624491 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba4922-466a-47a4-bb8e-a142e9457a01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:53.626175 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba4922-466a-47a4-bb8e-a142e9457a01 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:53.635810 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65305bf-d857-4283-956e-5717ee84d3d0 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:53.638755 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 867/1726] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:53.643025 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab686aea-1564-4878-a62e-6a523b3d4ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:53.643754 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab686aea-1564-4878-a62e-6a523b3d4ec0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:53.644209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab686aea-1564-4878-a62e-6a523b3d4ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:53.644490 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab686aea-1564-4878-a62e-6a523b3d4ec0 admin admin] PATH_INFO: `/v3/projects/0125cf348690425fa837bd688d0d17c6/users/801c59c279be48fab268c60f4aa9279f/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:53.663562 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab686aea-1564-4878-a62e-6a523b3d4ec0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0125cf348690425fa837bd688d0d17c6, user_id=801c59c279be48fab268c60f4aa9279f, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:53.667120 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab686aea-1564-4878-a62e-6a523b3d4ec0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:53.805187 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 860/1727] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:41:53 2022] PUT /identity/v3/projects/0125cf348690425fa837bd688d0d17c6/users/801c59c279be48fab268c60f4aa9279f/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:41:53.809175 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5f1c8-b23d-4e0a-83a6-290cbf21db80 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:53.810084 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5f1c8-b23d-4e0a-83a6-290cbf21db80 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:53.810517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5f1c8-b23d-4e0a-83a6-290cbf21db80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:53.846169 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-90c5f1c8-b23d-4e0a-83a6-290cbf21db80 None None] MFA Rules not processed for user `801c59c279be48fab268c60f4aa9279f`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:53.847621 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-90c5f1c8-b23d-4e0a-83a6-290cbf21db80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:53.951520 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab686aea-1564-4878-a62e-6a523b3d4ec0 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:53.953401 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ab686aea-1564-4878-a62e-6a523b3d4ec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:54.006106 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 868/1728] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:54.012793 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c5f1c8-b23d-4e0a-83a6-290cbf21db80 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:54.100627 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c5f1c8-b23d-4e0a-83a6-290cbf21db80 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:54.108794 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f4f05-915a-4a8b-9406-f803141d19f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:54.109424 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f4f05-915a-4a8b-9406-f803141d19f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:54.109810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f4f05-915a-4a8b-9406-f803141d19f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:54.115338 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f4f05-915a-4a8b-9406-f803141d19f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:54.115338 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137f4f05-915a-4a8b-9406-f803141d19f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:54.115750 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137f4f05-915a-4a8b-9406-f803141d19f6 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:54.121448 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab686aea-1564-4878-a62e-6a523b3d4ec0 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:54.128390 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364caca5-9e7a-4329-a77b-11760352d7c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:54.129885 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364caca5-9e7a-4329-a77b-11760352d7c1 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:54.130380 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364caca5-9e7a-4329-a77b-11760352d7c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:54.130817 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364caca5-9e7a-4329-a77b-11760352d7c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:54.133235 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-364caca5-9e7a-4329-a77b-11760352d7c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:54.149013 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364caca5-9e7a-4329-a77b-11760352d7c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:54.151272 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364caca5-9e7a-4329-a77b-11760352d7c1 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:54.164059 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 861/1729] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:54.209556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364caca5-9e7a-4329-a77b-11760352d7c1 service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:54.213466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-364caca5-9e7a-4329-a77b-11760352d7c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:54.227421 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 869/1730] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:41:54 2022] POST /identity/v3/projects => generated 377 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:54.232170 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f4f05-915a-4a8b-9406-f803141d19f6 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:54.267604 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364caca5-9e7a-4329-a77b-11760352d7c1 service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:54.409187 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f4f05-915a-4a8b-9406-f803141d19f6 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:54.427017 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58a562-b261-4e1e-949d-9cd22715b988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:54.429976 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58a562-b261-4e1e-949d-9cd22715b988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:54.433015 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef9bee-b46c-46fd-8562-351544e8f35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:54.433539 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58a562-b261-4e1e-949d-9cd22715b988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:54.433955 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58a562-b261-4e1e-949d-9cd22715b988 admin admin] PATH_INFO: `/v3/projects/cbb57fe80e9e404c923543980c601ccb` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:54.434728 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef9bee-b46c-46fd-8562-351544e8f35c admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:54.435965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef9bee-b46c-46fd-8562-351544e8f35c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:54.436982 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc58a562-b261-4e1e-949d-9cd22715b988 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbb57fe80e9e404c923543980c601ccb)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:54.437442 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef9bee-b46c-46fd-8562-351544e8f35c admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:54.439091 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ef9bee-b46c-46fd-8562-351544e8f35c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:54.443277 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ef9bee-b46c-46fd-8562-351544e8f35c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:54.446984 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc58a562-b261-4e1e-949d-9cd22715b988 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:54.458598 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:54.631540 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 870/1731] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:41:54 2022] POST /identity/v3/users => generated 289 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:54.632274 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef9bee-b46c-46fd-8562-351544e8f35c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:54.681861 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef9bee-b46c-46fd-8562-351544e8f35c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:54.690637 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-cc58a562-b261-4e1e-949d-9cd22715b988 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbb57fe80e9e404c923543980c601ccb', 'request_id': 'req-cc58a562-b261-4e1e-949d-9cd22715b988'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:54.705129 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-cc58a562-b261-4e1e-949d-9cd22715b988 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbb57fe80e9e404c923543980c601ccb', 'request_id': 'req-cc58a562-b261-4e1e-949d-9cd22715b988'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:54.769919 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc894b6-9289-4d33-b069-54f09b564f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:54.771114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc894b6-9289-4d33-b069-54f09b564f45 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:54.771260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc894b6-9289-4d33-b069-54f09b564f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:54.771486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc894b6-9289-4d33-b069-54f09b564f45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:54.772142 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc894b6-9289-4d33-b069-54f09b564f45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:54.773746 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc894b6-9289-4d33-b069-54f09b564f45 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:54.778827 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-cc58a562-b261-4e1e-949d-9cd22715b988 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbb57fe80e9e404c923543980c601ccb', 'request_id': 'req-cc58a562-b261-4e1e-949d-9cd22715b988'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:54.792393 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 871/1732] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:54 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:54.793962 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc894b6-9289-4d33-b069-54f09b564f45 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:54.802684 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-cc58a562-b261-4e1e-949d-9cd22715b988 admin admin] The token cache is being invalidate because project cbb57fe80e9e404c923543980c601ccb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:41:54.803917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-cc58a562-b261-4e1e-949d-9cd22715b988 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:54.804938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-cc58a562-b261-4e1e-949d-9cd22715b988 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:54.805908 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc894b6-9289-4d33-b069-54f09b564f45 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:54.808624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 862/1733] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:41:54 2022] DELETE /identity/v3/projects/cbb57fe80e9e404c923543980c601ccb => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:41:54.811370 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333df99-7b31-4982-a03d-3cd68afadc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:54.814303 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333df99-7b31-4982-a03d-3cd68afadc22 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:54.814303 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333df99-7b31-4982-a03d-3cd68afadc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:54.814303 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333df99-7b31-4982-a03d-3cd68afadc22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:54.814303 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333df99-7b31-4982-a03d-3cd68afadc22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:54.815125 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333df99-7b31-4982-a03d-3cd68afadc22 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:54.817212 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58a562-b261-4e1e-949d-9cd22715b988 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:54.818968 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cc58a562-b261-4e1e-949d-9cd22715b988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:54.839614 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 872/1734] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:54 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:54.841113 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333df99-7b31-4982-a03d-3cd68afadc22 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:54.842760 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58a562-b261-4e1e-949d-9cd22715b988 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:54.843967 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e333df99-7b31-4982-a03d-3cd68afadc22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:54.849061 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd996797-c7e8-42c8-a441-9d62dfd19549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:54.849732 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd996797-c7e8-42c8-a441-9d62dfd19549 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:54.850086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd996797-c7e8-42c8-a441-9d62dfd19549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:54.850294 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd996797-c7e8-42c8-a441-9d62dfd19549 admin admin] PATH_INFO: `/v3/users/06104c78dade47f7be89e4347390ee64` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:54.851548 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd996797-c7e8-42c8-a441-9d62dfd19549 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06104c78dade47f7be89e4347390ee64)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:54.854006 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd996797-c7e8-42c8-a441-9d62dfd19549 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:54.875085 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333df99-7b31-4982-a03d-3cd68afadc22 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:54.882327 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302efbe-a15b-4457-8211-a7957943f4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:54.883408 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302efbe-a15b-4457-8211-a7957943f4e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:54.890566 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302efbe-a15b-4457-8211-a7957943f4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:54.890566 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302efbe-a15b-4457-8211-a7957943f4e1 admin admin] PATH_INFO: `/v3/projects/4c497beedf944c469dd708dc69239b6f/users/7122ca675422412182def8cc10480b39/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:54.913985 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302efbe-a15b-4457-8211-a7957943f4e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c497beedf944c469dd708dc69239b6f, user_id=7122ca675422412182def8cc10480b39, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:54.917393 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302efbe-a15b-4457-8211-a7957943f4e1 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:55.003075 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 873/1735] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:41:54 2022] PUT /identity/v3/projects/4c497beedf944c469dd708dc69239b6f/users/7122ca675422412182def8cc10480b39/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:41:55.005703 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c374149-2517-4c65-a6be-5180915537aa None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:55.006688 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c374149-2517-4c65-a6be-5180915537aa None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:55.008990 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c374149-2517-4c65-a6be-5180915537aa None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:55.010594 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 874/1736] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 10 12:41:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:41:55.022316 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5c07f-f9ca-463d-a6a7-38d8217877d1 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:55.022863 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5c07f-f9ca-463d-a6a7-38d8217877d1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:55.023415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5c07f-f9ca-463d-a6a7-38d8217877d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:55.093021 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fd996797-c7e8-42c8-a441-9d62dfd19549 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06104c78dade47f7be89e4347390ee64'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:55.094269 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-fde5c07f-f9ca-463d-a6a7-38d8217877d1 None None] MFA Rules not processed for user `7122ca675422412182def8cc10480b39`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:55.095447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-fde5c07f-f9ca-463d-a6a7-38d8217877d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:55.190421 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fd996797-c7e8-42c8-a441-9d62dfd19549 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06104c78dade47f7be89e4347390ee64'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:55.191458 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fd996797-c7e8-42c8-a441-9d62dfd19549 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06104c78dade47f7be89e4347390ee64'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:55.193598 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fd996797-c7e8-42c8-a441-9d62dfd19549 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06104c78dade47f7be89e4347390ee64'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:55.216189 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fd996797-c7e8-42c8-a441-9d62dfd19549 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06104c78dade47f7be89e4347390ee64'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:55.229521 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 875/1737] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:55.231121 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c2b98-3f8c-44db-ad5e-2bed296de269 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:55.233976 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c2b98-3f8c-44db-ad5e-2bed296de269 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:55.233976 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c2b98-3f8c-44db-ad5e-2bed296de269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:55.253477 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:55.276464 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-760c2b98-3f8c-44db-ad5e-2bed296de269 None None] MFA Rules not processed for user `eb3f6deffb54467f8b923956ed2a6acf`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:55.277241 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 863/1738] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:41:54 2022] DELETE /identity/v3/users/06104c78dade47f7be89e4347390ee64 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:41:55.278895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4426795-b802-4ed5-96d2-7c702d6e42f6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:55.279314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4426795-b802-4ed5-96d2-7c702d6e42f6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:55.279420 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-760c2b98-3f8c-44db-ad5e-2bed296de269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:55.290624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4426795-b802-4ed5-96d2-7c702d6e42f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:55.376098 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a4426795-b802-4ed5-96d2-7c702d6e42f6 None None] MFA Rules not processed for user `c0ada1accf1742b393e8a12c60074b9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:55.380872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a4426795-b802-4ed5-96d2-7c702d6e42f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:55.450870 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 876/1739] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 10 12:41:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:55.454039 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c2b98-3f8c-44db-ad5e-2bed296de269 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:55.458468 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-760c2b98-3f8c-44db-ad5e-2bed296de269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:55.501691 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 864/1740] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:55.577779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c2b98-3f8c-44db-ad5e-2bed296de269 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:55.583966 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8238f-d138-4def-b5bd-9d20b0e426c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:55.584597 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8238f-d138-4def-b5bd-9d20b0e426c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:55.584881 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8238f-d138-4def-b5bd-9d20b0e426c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:55.585156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8238f-d138-4def-b5bd-9d20b0e426c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:55.585824 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8238f-d138-4def-b5bd-9d20b0e426c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:55.587779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8238f-d138-4def-b5bd-9d20b0e426c8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:55.696741 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4426795-b802-4ed5-96d2-7c702d6e42f6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:55.699930 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a4426795-b802-4ed5-96d2-7c702d6e42f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:55.708945 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 877/1741] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:41:55 2022] POST /identity/v3/projects => generated 377 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:55.715369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8238f-d138-4def-b5bd-9d20b0e426c8 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:55.752348 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4426795-b802-4ed5-96d2-7c702d6e42f6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:55.888137 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8238f-d138-4def-b5bd-9d20b0e426c8 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:55.900184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad45257-fd40-4f24-943b-2ce3e896bb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:55.901945 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad45257-fd40-4f24-943b-2ce3e896bb41 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:55.902821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad45257-fd40-4f24-943b-2ce3e896bb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:55.904340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad45257-fd40-4f24-943b-2ce3e896bb41 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:55.905678 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad45257-fd40-4f24-943b-2ce3e896bb41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:55.911803 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad45257-fd40-4f24-943b-2ce3e896bb41 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:55.915771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8accf-17a0-4c50-a55e-0c180f77ed46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:55.918743 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8accf-17a0-4c50-a55e-0c180f77ed46 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:55.919469 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8accf-17a0-4c50-a55e-0c180f77ed46 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:55.924682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8accf-17a0-4c50-a55e-0c180f77ed46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:55.955755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a8accf-17a0-4c50-a55e-0c180f77ed46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:55.959849 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a8accf-17a0-4c50-a55e-0c180f77ed46 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:55.975843 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 865/1742] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:56.022355 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409b6b6-acb7-4d11-a41e-1230824ef231 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:56.022641 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409b6b6-acb7-4d11-a41e-1230824ef231 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:56.022916 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409b6b6-acb7-4d11-a41e-1230824ef231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:56.050195 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 878/1743] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:41:55 2022] POST /identity/v3/users => generated 295 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:56.058639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad45257-fd40-4f24-943b-2ce3e896bb41 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:56.072279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5409b6b6-acb7-4d11-a41e-1230824ef231 None None] MFA Rules not processed for user `570c2a77b862411083b9444f8292192b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:56.076528 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad45257-fd40-4f24-943b-2ce3e896bb41 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:56.076736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5409b6b6-acb7-4d11-a41e-1230824ef231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:56.102395 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dad1f5-e826-411d-8ace-5a1a00907745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:56.103421 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dad1f5-e826-411d-8ace-5a1a00907745 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:56.103931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dad1f5-e826-411d-8ace-5a1a00907745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:56.106277 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dad1f5-e826-411d-8ace-5a1a00907745 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:56.107378 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dad1f5-e826-411d-8ace-5a1a00907745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:56.111714 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dad1f5-e826-411d-8ace-5a1a00907745 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:56.136388 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 879/1744] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:56.137852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dad1f5-e826-411d-8ace-5a1a00907745 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:56.154100 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dad1f5-e826-411d-8ace-5a1a00907745 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:56.160598 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80e97e-070b-45b2-b16a-0346d47a50f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:56.161972 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80e97e-070b-45b2-b16a-0346d47a50f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:56.163237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80e97e-070b-45b2-b16a-0346d47a50f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:56.163237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80e97e-070b-45b2-b16a-0346d47a50f2 admin admin] PATH_INFO: `/v3/projects/734c44a7ef654afb8a2ebd83f2f67bf3/users/fb689d487add49b193f0a7845efca4fc/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:56.197644 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80e97e-070b-45b2-b16a-0346d47a50f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=734c44a7ef654afb8a2ebd83f2f67bf3, user_id=fb689d487add49b193f0a7845efca4fc, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:56.203409 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80e97e-070b-45b2-b16a-0346d47a50f2 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:56.264462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 866/1745] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:56.297648 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 880/1746] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:41:56 2022] PUT /identity/v3/projects/734c44a7ef654afb8a2ebd83f2f67bf3/users/fb689d487add49b193f0a7845efca4fc/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:41:56.305701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409b6b6-acb7-4d11-a41e-1230824ef231 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:56.319746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409b6b6-acb7-4d11-a41e-1230824ef231 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:56.421048 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60529d6c-7006-4634-b42e-b849ea199b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:56.424583 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60529d6c-7006-4634-b42e-b849ea199b10 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:56.424893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60529d6c-7006-4634-b42e-b849ea199b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:56.425168 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60529d6c-7006-4634-b42e-b849ea199b10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:56.426243 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60529d6c-7006-4634-b42e-b849ea199b10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:56.429163 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60529d6c-7006-4634-b42e-b849ea199b10 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:56.448914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 867/1747] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:56.451765 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80e97e-070b-45b2-b16a-0346d47a50f2 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:56.466681 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80e97e-070b-45b2-b16a-0346d47a50f2 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:56.480557 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b47ef5-4616-4711-a413-45728bd8a32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:56.481462 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b47ef5-4616-4711-a413-45728bd8a32d admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:56.481817 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b47ef5-4616-4711-a413-45728bd8a32d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:56.482167 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b47ef5-4616-4711-a413-45728bd8a32d admin admin] PATH_INFO: `/v3/projects/734c44a7ef654afb8a2ebd83f2f67bf3/users/fb689d487add49b193f0a7845efca4fc/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:56.484281 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b47ef5-4616-4711-a413-45728bd8a32d admin admin] RBAC: Authorizing `identity:create_grant(project_id=734c44a7ef654afb8a2ebd83f2f67bf3, user_id=fb689d487add49b193f0a7845efca4fc, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:56.487413 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b47ef5-4616-4711-a413-45728bd8a32d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:56.513265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 881/1748] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:41:56 2022] PUT /identity/v3/projects/734c44a7ef654afb8a2ebd83f2f67bf3/users/fb689d487add49b193f0a7845efca4fc/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:41:56.514779 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e50c5a-49d8-480d-8295-df4772d5cd5b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:56.515250 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e50c5a-49d8-480d-8295-df4772d5cd5b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:56.516677 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e50c5a-49d8-480d-8295-df4772d5cd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:56.593111 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f0e50c5a-49d8-480d-8295-df4772d5cd5b None None] MFA Rules not processed for user `fb689d487add49b193f0a7845efca4fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:56.596810 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f0e50c5a-49d8-480d-8295-df4772d5cd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:56.693622 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b47ef5-4616-4711-a413-45728bd8a32d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:56.696138 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f1b47ef5-4616-4711-a413-45728bd8a32d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:56.876668 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 868/1749] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:56 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:41:56.898870 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e50c5a-49d8-480d-8295-df4772d5cd5b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:56.901210 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f0e50c5a-49d8-480d-8295-df4772d5cd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:57.007406 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b47ef5-4616-4711-a413-45728bd8a32d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:57.019763 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:57.030668 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e7c3c8-259b-4205-b7e4-2a7dca5b55d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:57.031589 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7c3c8-259b-4205-b7e4-2a7dca5b55d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:57.031812 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7c3c8-259b-4205-b7e4-2a7dca5b55d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:57.032084 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7c3c8-259b-4205-b7e4-2a7dca5b55d2 admin admin] PATH_INFO: `/v3/projects/97b2bc810e014edf9b3e14557a34eb7e` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:57.033270 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e7c3c8-259b-4205-b7e4-2a7dca5b55d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97b2bc810e014edf9b3e14557a34eb7e)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:57.042605 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e7c3c8-259b-4205-b7e4-2a7dca5b55d2 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:57.048163 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:57.092051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e50c5a-49d8-480d-8295-df4772d5cd5b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:57.108673 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6dd9bb-70cb-41c1-b237-a73244f7bae5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:57.113273 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6dd9bb-70cb-41c1-b237-a73244f7bae5 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:57.113273 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6dd9bb-70cb-41c1-b237-a73244f7bae5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:57.113273 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6dd9bb-70cb-41c1-b237-a73244f7bae5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:57.142911 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6dd9bb-70cb-41c1-b237-a73244f7bae5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:57.156943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6dd9bb-70cb-41c1-b237-a73244f7bae5 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:57.183062 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 869/1750] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:56 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:41:57.300648 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-62e7c3c8-259b-4205-b7e4-2a7dca5b55d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97b2bc810e014edf9b3e14557a34eb7e', 'request_id': 'req-62e7c3c8-259b-4205-b7e4-2a7dca5b55d2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:57.331418 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-62e7c3c8-259b-4205-b7e4-2a7dca5b55d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97b2bc810e014edf9b3e14557a34eb7e', 'request_id': 'req-62e7c3c8-259b-4205-b7e4-2a7dca5b55d2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:57.366307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6dd9bb-70cb-41c1-b237-a73244f7bae5 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:57.376997 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6dd9bb-70cb-41c1-b237-a73244f7bae5 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:57.442062 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-62e7c3c8-259b-4205-b7e4-2a7dca5b55d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97b2bc810e014edf9b3e14557a34eb7e', 'request_id': 'req-62e7c3c8-259b-4205-b7e4-2a7dca5b55d2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:57.466480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-62e7c3c8-259b-4205-b7e4-2a7dca5b55d2 admin admin] The token cache is being invalidate because project 97b2bc810e014edf9b3e14557a34eb7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:41:57.466777 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-62e7c3c8-259b-4205-b7e4-2a7dca5b55d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:57.468013 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-62e7c3c8-259b-4205-b7e4-2a7dca5b55d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:57.478308 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 882/1751] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:41:56 2022] DELETE /identity/v3/projects/97b2bc810e014edf9b3e14557a34eb7e => generated 0 bytes in 784 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:41:57.485989 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29aa19-16b2-499f-994f-3d94f222a728 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:57.490801 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29aa19-16b2-499f-994f-3d94f222a728 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:57.490801 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29aa19-16b2-499f-994f-3d94f222a728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:57.491169 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8279bf9-bd3c-443a-9781-0a40af492a86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:57.491169 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8279bf9-bd3c-443a-9781-0a40af492a86 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:57.491169 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8279bf9-bd3c-443a-9781-0a40af492a86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:57.491482 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8279bf9-bd3c-443a-9781-0a40af492a86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:57.497616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b8279bf9-bd3c-443a-9781-0a40af492a86 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:57.567940 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ee29aa19-16b2-499f-994f-3d94f222a728 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:57.572094 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ee29aa19-16b2-499f-994f-3d94f222a728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:57.596684 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8279bf9-bd3c-443a-9781-0a40af492a86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:57.598537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8279bf9-bd3c-443a-9781-0a40af492a86 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:57.619462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 870/1752] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:57.764818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 883/1753] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:57.772935 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f162bb5-76fc-4647-b6e9-e0011eec56bf None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:57.773233 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f162bb5-76fc-4647-b6e9-e0011eec56bf None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:57.773564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f162bb5-76fc-4647-b6e9-e0011eec56bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:57.849688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0f162bb5-76fc-4647-b6e9-e0011eec56bf None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:57.857825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0f162bb5-76fc-4647-b6e9-e0011eec56bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:57.887601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 871/1754] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:57.888882 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29aa19-16b2-499f-994f-3d94f222a728 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:57.910941 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29aa19-16b2-499f-994f-3d94f222a728 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:57.924466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c6fe4-1126-44f8-9f0f-52682289ee8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:57.925106 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c6fe4-1126-44f8-9f0f-52682289ee8b admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:57.925391 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c6fe4-1126-44f8-9f0f-52682289ee8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:57.925672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c6fe4-1126-44f8-9f0f-52682289ee8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:57.926509 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c6fe4-1126-44f8-9f0f-52682289ee8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:41:57.926771 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c6fe4-1126-44f8-9f0f-52682289ee8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:57.932099 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c6fe4-1126-44f8-9f0f-52682289ee8b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:57.986731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 884/1755] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:41:57.990125 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f162bb5-76fc-4647-b6e9-e0011eec56bf None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:58.022398 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f162bb5-76fc-4647-b6e9-e0011eec56bf None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:58.036273 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b8cf0-1d5a-4b90-bf0e-4e1a1149cd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:58.037372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b8cf0-1d5a-4b90-bf0e-4e1a1149cd8c admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:58.037819 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b8cf0-1d5a-4b90-bf0e-4e1a1149cd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:58.038440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b8cf0-1d5a-4b90-bf0e-4e1a1149cd8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:58.040053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b8cf0-1d5a-4b90-bf0e-4e1a1149cd8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:58.042231 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b8cf0-1d5a-4b90-bf0e-4e1a1149cd8c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:58.176738 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 872/1756] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:41:57 2022] POST /identity/v3/projects => generated 379 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:41:58.181486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c6fe4-1126-44f8-9f0f-52682289ee8b admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:58.200925 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c6fe4-1126-44f8-9f0f-52682289ee8b admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:58.304634 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:58.372398 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:58.394221 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb680c0-a7a9-46e5-b9a7-ddc88de4bf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:58.394743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb680c0-a7a9-46e5-b9a7-ddc88de4bf05 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:58.396224 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb680c0-a7a9-46e5-b9a7-ddc88de4bf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:58.396224 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb680c0-a7a9-46e5-b9a7-ddc88de4bf05 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:58.396767 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb680c0-a7a9-46e5-b9a7-ddc88de4bf05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:58.399633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb680c0-a7a9-46e5-b9a7-ddc88de4bf05 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:58.535236 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 885/1757] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:41:58 2022] POST /identity/v3/users => generated 290 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:58.539102 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b8cf0-1d5a-4b90-bf0e-4e1a1149cd8c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:58.563127 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b8cf0-1d5a-4b90-bf0e-4e1a1149cd8c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:58.569389 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5d2c0-6c49-4c9d-b86b-3efe5ffb16ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:58.570042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5d2c0-6c49-4c9d-b86b-3efe5ffb16ce admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:58.570486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5d2c0-6c49-4c9d-b86b-3efe5ffb16ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:58.572547 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5d2c0-6c49-4c9d-b86b-3efe5ffb16ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:58.573235 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b5d2c0-6c49-4c9d-b86b-3efe5ffb16ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:58.588503 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b5d2c0-6c49-4c9d-b86b-3efe5ffb16ce admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:58.625250 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb680c0-a7a9-46e5-b9a7-ddc88de4bf05 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:58.626047 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 873/1758] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:41:58.638894 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb680c0-a7a9-46e5-b9a7-ddc88de4bf05 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:58.644171 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788b2af-e718-4472-9186-10363d7dac44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:58.645693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788b2af-e718-4472-9186-10363d7dac44 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:58.646361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788b2af-e718-4472-9186-10363d7dac44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:58.646897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788b2af-e718-4472-9186-10363d7dac44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:58.647925 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788b2af-e718-4472-9186-10363d7dac44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:58.651257 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788b2af-e718-4472-9186-10363d7dac44 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:58.669859 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 886/1759] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:41:58.671072 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5d2c0-6c49-4c9d-b86b-3efe5ffb16ce admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:58.683845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5d2c0-6c49-4c9d-b86b-3efe5ffb16ce admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:58.696569 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0587d47-feb6-4536-a7dc-d95dba283ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:58.697220 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0587d47-feb6-4536-a7dc-d95dba283ab9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:58.697552 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0587d47-feb6-4536-a7dc-d95dba283ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:58.697906 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0587d47-feb6-4536-a7dc-d95dba283ab9 admin admin] PATH_INFO: `/v3/projects/9530deaea206450d96772274828cca3b/users/c5e282c3da474ff09c49163969392b2c/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:58.728701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0587d47-feb6-4536-a7dc-d95dba283ab9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9530deaea206450d96772274828cca3b, user_id=c5e282c3da474ff09c49163969392b2c, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:58.735067 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0587d47-feb6-4536-a7dc-d95dba283ab9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:58.821764 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 874/1760] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:41:58 2022] PUT /identity/v3/projects/9530deaea206450d96772274828cca3b/users/c5e282c3da474ff09c49163969392b2c/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:41:58.833952 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bcb0d4-2c1d-4d08-865e-2861777d5d0a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:58.834239 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bcb0d4-2c1d-4d08-865e-2861777d5d0a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:58.834522 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bcb0d4-2c1d-4d08-865e-2861777d5d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:58.921619 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-51bcb0d4-2c1d-4d08-865e-2861777d5d0a None None] MFA Rules not processed for user `c5e282c3da474ff09c49163969392b2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:58.922897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-51bcb0d4-2c1d-4d08-865e-2861777d5d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:59.071559 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 887/1761] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:41:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:59.076383 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0587d47-feb6-4536-a7dc-d95dba283ab9 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:59.205358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0587d47-feb6-4536-a7dc-d95dba283ab9 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:59.215068 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96be761-a369-4b6b-b418-00726b9e3f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:59.215281 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96be761-a369-4b6b-b418-00726b9e3f18 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:59.215565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96be761-a369-4b6b-b418-00726b9e3f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:59.215834 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96be761-a369-4b6b-b418-00726b9e3f18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:59.217000 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96be761-a369-4b6b-b418-00726b9e3f18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:59.218891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96be761-a369-4b6b-b418-00726b9e3f18 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:59.329901 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 875/1762] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:41:59 2022] POST /identity/v3/projects => generated 381 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:59.331040 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bcb0d4-2c1d-4d08-865e-2861777d5d0a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:59.352465 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bcb0d4-2c1d-4d08-865e-2861777d5d0a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:59.446427 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63f69a-567b-43c8-a8dc-4cde1127f8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:59.447474 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63f69a-567b-43c8-a8dc-4cde1127f8c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:59.447764 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63f69a-567b-43c8-a8dc-4cde1127f8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:59.448043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63f69a-567b-43c8-a8dc-4cde1127f8c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:59.450258 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c63f69a-567b-43c8-a8dc-4cde1127f8c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:59.463164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c63f69a-567b-43c8-a8dc-4cde1127f8c3 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:59.577135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 888/1763] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:41:59 2022] POST /identity/v3/users => generated 297 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:41:59.580623 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96be761-a369-4b6b-b418-00726b9e3f18 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:59.606348 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96be761-a369-4b6b-b418-00726b9e3f18 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:59.612149 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17570222-db8d-44f7-8efe-6ac8514f172a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:59.613085 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17570222-db8d-44f7-8efe-6ac8514f172a admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:59.613376 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17570222-db8d-44f7-8efe-6ac8514f172a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:59.614152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17570222-db8d-44f7-8efe-6ac8514f172a admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:59.615613 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17570222-db8d-44f7-8efe-6ac8514f172a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:59.617696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17570222-db8d-44f7-8efe-6ac8514f172a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:59.659235 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 876/1764] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:41:59.661560 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63f69a-567b-43c8-a8dc-4cde1127f8c3 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:59.673986 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63f69a-567b-43c8-a8dc-4cde1127f8c3 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:59.682367 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b51c76-87de-4b68-9c8d-44b99ac42c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:59.683351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b51c76-87de-4b68-9c8d-44b99ac42c6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:59.683709 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b51c76-87de-4b68-9c8d-44b99ac42c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:59.684222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b51c76-87de-4b68-9c8d-44b99ac42c6f admin admin] PATH_INFO: `/v3/projects/c62ca1a705f34f12b1d7b7190c9cbf3a/users/eb1f6270711447f08fe164404767f82d/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:59.711269 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b51c76-87de-4b68-9c8d-44b99ac42c6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c62ca1a705f34f12b1d7b7190c9cbf3a, user_id=eb1f6270711447f08fe164404767f82d, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:59.729533 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b51c76-87de-4b68-9c8d-44b99ac42c6f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:59.865152 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 889/1765] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:41:59 2022] PUT /identity/v3/projects/c62ca1a705f34f12b1d7b7190c9cbf3a/users/eb1f6270711447f08fe164404767f82d/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:41:59.865428 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17570222-db8d-44f7-8efe-6ac8514f172a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:59.896230 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17570222-db8d-44f7-8efe-6ac8514f172a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:00.024249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495eae3-630c-49e2-93fd-211e23ad1254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:00.025207 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495eae3-630c-49e2-93fd-211e23ad1254 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:00.025495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495eae3-630c-49e2-93fd-211e23ad1254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:00.025870 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495eae3-630c-49e2-93fd-211e23ad1254 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:00.026420 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2495eae3-630c-49e2-93fd-211e23ad1254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:00.028312 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2495eae3-630c-49e2-93fd-211e23ad1254 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:00.050424 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 877/1766] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:00.053940 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b51c76-87de-4b68-9c8d-44b99ac42c6f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:00.092432 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b51c76-87de-4b68-9c8d-44b99ac42c6f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:00.101039 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:00.111919 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59d143-9726-45bf-a1d8-a2f00ec52989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:00.111919 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59d143-9726-45bf-a1d8-a2f00ec52989 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:00.111919 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59d143-9726-45bf-a1d8-a2f00ec52989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:00.111919 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59d143-9726-45bf-a1d8-a2f00ec52989 admin admin] PATH_INFO: `/v3/projects/c62ca1a705f34f12b1d7b7190c9cbf3a/users/eb1f6270711447f08fe164404767f82d/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:00.112817 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59d143-9726-45bf-a1d8-a2f00ec52989 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c62ca1a705f34f12b1d7b7190c9cbf3a, user_id=eb1f6270711447f08fe164404767f82d, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:00.117251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59d143-9726-45bf-a1d8-a2f00ec52989 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:00.144750 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 890/1767] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:42:00 2022] PUT /identity/v3/projects/c62ca1a705f34f12b1d7b7190c9cbf3a/users/eb1f6270711447f08fe164404767f82d/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:42:00.145979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7b046-47d2-4a4a-b6f3-d3235034ff61 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:00.146225 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7b046-47d2-4a4a-b6f3-d3235034ff61 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:00.146504 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7b046-47d2-4a4a-b6f3-d3235034ff61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:00.230448 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b2c7b046-47d2-4a4a-b6f3-d3235034ff61 None None] MFA Rules not processed for user `eb1f6270711447f08fe164404767f82d`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:00.240991 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b2c7b046-47d2-4a4a-b6f3-d3235034ff61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:00.431692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 878/1768] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:00 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:42:00.593189 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59d143-9726-45bf-a1d8-a2f00ec52989 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:00.596385 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5b59d143-9726-45bf-a1d8-a2f00ec52989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:00.696905 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59d143-9726-45bf-a1d8-a2f00ec52989 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:00.719257 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019c41b-32e0-4591-8766-85b0861230cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:00.720218 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019c41b-32e0-4591-8766-85b0861230cc service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:00.720679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019c41b-32e0-4591-8766-85b0861230cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:00.721708 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019c41b-32e0-4591-8766-85b0861230cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:00.755002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4019c41b-32e0-4591-8766-85b0861230cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:00.756676 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4019c41b-32e0-4591-8766-85b0861230cc service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:00.769534 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 891/1769] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:00 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:42:00.842343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:01.339732 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:01.425451 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:01.836949 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:02.601483 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c7b046-47d2-4a4a-b6f3-d3235034ff61 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:02.612663 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c7b046-47d2-4a4a-b6f3-d3235034ff61 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:02.618280 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f36fb-ddda-49dd-ab00-88588a5ddcaf tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb1f6270711447f08fe164404767f82d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c62ca1a705f34f12b1d7b7190c9cbf3a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:02.618915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f36fb-ddda-49dd-ab00-88588a5ddcaf tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:02.619219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f36fb-ddda-49dd-ab00-88588a5ddcaf tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:02.619499 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f36fb-ddda-49dd-ab00-88588a5ddcaf tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:02.620158 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f36fb-ddda-49dd-ab00-88588a5ddcaf tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:42:02.620463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f36fb-ddda-49dd-ab00-88588a5ddcaf tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:02.622118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f36fb-ddda-49dd-ab00-88588a5ddcaf tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:02.636361 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 879/1770] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:42:02.640551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019c41b-32e0-4591-8766-85b0861230cc service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:02.650321 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019c41b-32e0-4591-8766-85b0861230cc service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:02.655931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd96a4b6-5645-43f1-9d8b-217c4b4131b0 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb1f6270711447f08fe164404767f82d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c62ca1a705f34f12b1d7b7190c9cbf3a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:02.656422 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96a4b6-5645-43f1-9d8b-217c4b4131b0 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:02.656709 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96a4b6-5645-43f1-9d8b-217c4b4131b0 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:02.656987 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96a4b6-5645-43f1-9d8b-217c4b4131b0 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:02.658648 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd96a4b6-5645-43f1-9d8b-217c4b4131b0 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:02.660722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd96a4b6-5645-43f1-9d8b-217c4b4131b0 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:02.759928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 892/1771] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:42:02 2022] POST /identity/v3/projects => generated 355 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:42:02.764183 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f36fb-ddda-49dd-ab00-88588a5ddcaf tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:02.775359 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f36fb-ddda-49dd-ab00-88588a5ddcaf tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:02.857759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a7db4-5c55-4dd8-987a-b3cb2ff05c24 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb1f6270711447f08fe164404767f82d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c62ca1a705f34f12b1d7b7190c9cbf3a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:02.858823 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a7db4-5c55-4dd8-987a-b3cb2ff05c24 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:02.859166 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a7db4-5c55-4dd8-987a-b3cb2ff05c24 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:02.859552 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a7db4-5c55-4dd8-987a-b3cb2ff05c24 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:02.861046 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a7db4-5c55-4dd8-987a-b3cb2ff05c24 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:42:02.861494 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a7db4-5c55-4dd8-987a-b3cb2ff05c24 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:02.864110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a7db4-5c55-4dd8-987a-b3cb2ff05c24 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:02.883715 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 880/1772] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:42:02.940405 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd96a4b6-5645-43f1-9d8b-217c4b4131b0 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:02.951852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd96a4b6-5645-43f1-9d8b-217c4b4131b0 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:02.957239 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ba6d0-87c4-40f1-b110-01a77b14ca76 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb1f6270711447f08fe164404767f82d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c62ca1a705f34f12b1d7b7190c9cbf3a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:02.958080 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ba6d0-87c4-40f1-b110-01a77b14ca76 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:02.958392 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ba6d0-87c4-40f1-b110-01a77b14ca76 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:02.958651 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ba6d0-87c4-40f1-b110-01a77b14ca76 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] PATH_INFO: `/v3/projects/8072881f95064260a5f1586e314ab31c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:02.959902 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ba6d0-87c4-40f1-b110-01a77b14ca76 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8072881f95064260a5f1586e314ab31c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:02.961904 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ba6d0-87c4-40f1-b110-01a77b14ca76 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:03.110980 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:03.131109 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-391ba6d0-87c4-40f1-b110-01a77b14ca76 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8072881f95064260a5f1586e314ab31c', 'request_id': 'req-391ba6d0-87c4-40f1-b110-01a77b14ca76'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:03.138049 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-391ba6d0-87c4-40f1-b110-01a77b14ca76 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8072881f95064260a5f1586e314ab31c', 'request_id': 'req-391ba6d0-87c4-40f1-b110-01a77b14ca76'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:03.234568 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-391ba6d0-87c4-40f1-b110-01a77b14ca76 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8072881f95064260a5f1586e314ab31c', 'request_id': 'req-391ba6d0-87c4-40f1-b110-01a77b14ca76'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:03.247865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-391ba6d0-87c4-40f1-b110-01a77b14ca76 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] The token cache is being invalidate because project 8072881f95064260a5f1586e314ab31c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:42:03.248161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-391ba6d0-87c4-40f1-b110-01a77b14ca76 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:03.248849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-391ba6d0-87c4-40f1-b110-01a77b14ca76 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:03.250561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 893/1773] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:42:02 2022] DELETE /identity/v3/projects/8072881f95064260a5f1586e314ab31c => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:03.448360 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a7db4-5c55-4dd8-987a-b3cb2ff05c24 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:03.450306 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9d4a7db4-5c55-4dd8-987a-b3cb2ff05c24 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:03.525247 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a7db4-5c55-4dd8-987a-b3cb2ff05c24 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:03.557867 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:03.587841 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d58ac3-0afe-43e6-8271-55a77d89faef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:03.588482 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d58ac3-0afe-43e6-8271-55a77d89faef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:03.588759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d58ac3-0afe-43e6-8271-55a77d89faef admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:03.589092 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d58ac3-0afe-43e6-8271-55a77d89faef admin admin] PATH_INFO: `/v3/users/801c59c279be48fab268c60f4aa9279f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:03.590145 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d58ac3-0afe-43e6-8271-55a77d89faef admin admin] RBAC: Authorizing `identity:delete_user(user_id=801c59c279be48fab268c60f4aa9279f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:03.592093 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d58ac3-0afe-43e6-8271-55a77d89faef admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:03.802147 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-72d58ac3-0afe-43e6-8271-55a77d89faef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '801c59c279be48fab268c60f4aa9279f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:03.889115 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-72d58ac3-0afe-43e6-8271-55a77d89faef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '801c59c279be48fab268c60f4aa9279f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:03.889574 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-72d58ac3-0afe-43e6-8271-55a77d89faef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '801c59c279be48fab268c60f4aa9279f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:03.890212 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-72d58ac3-0afe-43e6-8271-55a77d89faef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '801c59c279be48fab268c60f4aa9279f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:03.892162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ba6d0-87c4-40f1-b110-01a77b14ca76 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:03.894269 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-391ba6d0-87c4-40f1-b110-01a77b14ca76 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:03.899375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-72d58ac3-0afe-43e6-8271-55a77d89faef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '801c59c279be48fab268c60f4aa9279f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:03.910460 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ba6d0-87c4-40f1-b110-01a77b14ca76 tempest-VolumeQuotaClassesTest-2097209147 tempest-VolumeQuotaClassesTest-2097209147-project-admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:03.915751 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d52bc-9dd5-4160-9fab-9a090eda6113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:03.916592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d52bc-9dd5-4160-9fab-9a090eda6113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:03.916886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d52bc-9dd5-4160-9fab-9a090eda6113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:03.917168 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d52bc-9dd5-4160-9fab-9a090eda6113 admin admin] PATH_INFO: `/v3/users/c5e282c3da474ff09c49163969392b2c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:03.918321 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7d52bc-9dd5-4160-9fab-9a090eda6113 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5e282c3da474ff09c49163969392b2c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:03.920337 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7d52bc-9dd5-4160-9fab-9a090eda6113 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:03.921613 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 881/1774] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:42:03 2022] DELETE /identity/v3/users/801c59c279be48fab268c60f4aa9279f => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:03.929152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679dac17-2b2e-4b65-813d-43add95589e9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:03.929594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679dac17-2b2e-4b65-813d-43add95589e9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:03.929882 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679dac17-2b2e-4b65-813d-43add95589e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:03.951212 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-679dac17-2b2e-4b65-813d-43add95589e9 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:03.952566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-679dac17-2b2e-4b65-813d-43add95589e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:03.961848 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:04.023375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 882/1775] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:42:04.035408 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679dac17-2b2e-4b65-813d-43add95589e9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:04.037407 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-679dac17-2b2e-4b65-813d-43add95589e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:04.121782 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679dac17-2b2e-4b65-813d-43add95589e9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:04.125479 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07617c99-ad92-439a-846a-55a236df44d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:04.126092 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07617c99-ad92-439a-846a-55a236df44d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:04.126371 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07617c99-ad92-439a-846a-55a236df44d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:04.126747 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07617c99-ad92-439a-846a-55a236df44d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:04.133212 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:04.139241 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07617c99-ad92-439a-846a-55a236df44d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:04.140223 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07617c99-ad92-439a-846a-55a236df44d2 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:04.158222 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 883/1776] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:42:04.167195 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-da7d52bc-9dd5-4160-9fab-9a090eda6113 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5e282c3da474ff09c49163969392b2c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:04.170983 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-da7d52bc-9dd5-4160-9fab-9a090eda6113 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5e282c3da474ff09c49163969392b2c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:04.275819 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-da7d52bc-9dd5-4160-9fab-9a090eda6113 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5e282c3da474ff09c49163969392b2c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:04.276877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-da7d52bc-9dd5-4160-9fab-9a090eda6113 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5e282c3da474ff09c49163969392b2c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:04.278638 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-da7d52bc-9dd5-4160-9fab-9a090eda6113 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5e282c3da474ff09c49163969392b2c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:04.318228 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 894/1777] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:42:03 2022] DELETE /identity/v3/users/c5e282c3da474ff09c49163969392b2c => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:04.322585 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3e07d-8d88-44c7-83e2-37308a5f42af None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:04.323001 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3e07d-8d88-44c7-83e2-37308a5f42af None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:04.323301 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3e07d-8d88-44c7-83e2-37308a5f42af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:04.343879 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c0a3e07d-8d88-44c7-83e2-37308a5f42af None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:04.345205 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c0a3e07d-8d88-44c7-83e2-37308a5f42af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:04.428504 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 884/1778] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:04.439813 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d52bc-9dd5-4160-9fab-9a090eda6113 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:04.443057 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-da7d52bc-9dd5-4160-9fab-9a090eda6113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:04.528769 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d52bc-9dd5-4160-9fab-9a090eda6113 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:04.537655 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35470a-179a-4d96-9ce7-af07fec24029 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:04.538809 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35470a-179a-4d96-9ce7-af07fec24029 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:04.538809 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35470a-179a-4d96-9ce7-af07fec24029 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:04.539016 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35470a-179a-4d96-9ce7-af07fec24029 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:04.550125 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35470a-179a-4d96-9ce7-af07fec24029 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:04.552789 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35470a-179a-4d96-9ce7-af07fec24029 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:04.563917 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 895/1779] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:42:04.780576 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3e07d-8d88-44c7-83e2-37308a5f42af None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:04.782201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c0a3e07d-8d88-44c7-83e2-37308a5f42af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:04.795568 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3e07d-8d88-44c7-83e2-37308a5f42af None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:04.800304 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f5d9b-643c-4e8e-b0f2-5cff48d809ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:04.800940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f5d9b-643c-4e8e-b0f2-5cff48d809ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:04.801765 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f5d9b-643c-4e8e-b0f2-5cff48d809ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:04.802116 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f5d9b-643c-4e8e-b0f2-5cff48d809ec admin admin] PATH_INFO: `/v3/projects/0125cf348690425fa837bd688d0d17c6` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:04.803334 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f5d9b-643c-4e8e-b0f2-5cff48d809ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=0125cf348690425fa837bd688d0d17c6)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:04.805281 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f5d9b-643c-4e8e-b0f2-5cff48d809ec admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:04.809559 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:04.958691 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3f8f5d9b-643c-4e8e-b0f2-5cff48d809ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0125cf348690425fa837bd688d0d17c6', 'request_id': 'req-3f8f5d9b-643c-4e8e-b0f2-5cff48d809ec'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:04.964862 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3f8f5d9b-643c-4e8e-b0f2-5cff48d809ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0125cf348690425fa837bd688d0d17c6', 'request_id': 'req-3f8f5d9b-643c-4e8e-b0f2-5cff48d809ec'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:05.080725 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3f8f5d9b-643c-4e8e-b0f2-5cff48d809ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0125cf348690425fa837bd688d0d17c6', 'request_id': 'req-3f8f5d9b-643c-4e8e-b0f2-5cff48d809ec'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:05.090731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35470a-179a-4d96-9ce7-af07fec24029 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:05.092166 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-fe35470a-179a-4d96-9ce7-af07fec24029 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:05.100547 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3f8f5d9b-643c-4e8e-b0f2-5cff48d809ec admin admin] The token cache is being invalidate because project 0125cf348690425fa837bd688d0d17c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:42:05.100928 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3f8f5d9b-643c-4e8e-b0f2-5cff48d809ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:05.101578 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3f8f5d9b-643c-4e8e-b0f2-5cff48d809ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:05.110060 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35470a-179a-4d96-9ce7-af07fec24029 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:05.115069 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 885/1780] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:42:04 2022] DELETE /identity/v3/projects/0125cf348690425fa837bd688d0d17c6 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:05.118481 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f5d9b-643c-4e8e-b0f2-5cff48d809ec admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:05.121864 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3f8f5d9b-643c-4e8e-b0f2-5cff48d809ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:05.178896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb5520d-bebe-44f0-ab62-26353912b99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:05.179588 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5520d-bebe-44f0-ab62-26353912b99d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:05.179874 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5520d-bebe-44f0-ab62-26353912b99d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:05.180118 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5520d-bebe-44f0-ab62-26353912b99d admin admin] PATH_INFO: `/v3/projects/9530deaea206450d96772274828cca3b` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:05.181272 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb5520d-bebe-44f0-ab62-26353912b99d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9530deaea206450d96772274828cca3b)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:05.183253 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb5520d-bebe-44f0-ab62-26353912b99d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:05.195368 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f5d9b-643c-4e8e-b0f2-5cff48d809ec admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:05.200478 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492fefa-9c21-4cd6-a8bd-89242c2a1b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:05.201121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492fefa-9c21-4cd6-a8bd-89242c2a1b13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:05.201408 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492fefa-9c21-4cd6-a8bd-89242c2a1b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:05.201687 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492fefa-9c21-4cd6-a8bd-89242c2a1b13 admin admin] PATH_INFO: `/v3/users/7122ca675422412182def8cc10480b39` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:05.202852 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492fefa-9c21-4cd6-a8bd-89242c2a1b13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7122ca675422412182def8cc10480b39)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:05.205042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492fefa-9c21-4cd6-a8bd-89242c2a1b13 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:05.338186 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-dcb5520d-bebe-44f0-ab62-26353912b99d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9530deaea206450d96772274828cca3b', 'request_id': 'req-dcb5520d-bebe-44f0-ab62-26353912b99d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:05.346772 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-dcb5520d-bebe-44f0-ab62-26353912b99d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9530deaea206450d96772274828cca3b', 'request_id': 'req-dcb5520d-bebe-44f0-ab62-26353912b99d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:05.437080 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-dcb5520d-bebe-44f0-ab62-26353912b99d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9530deaea206450d96772274828cca3b', 'request_id': 'req-dcb5520d-bebe-44f0-ab62-26353912b99d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:05.452646 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-dcb5520d-bebe-44f0-ab62-26353912b99d admin admin] The token cache is being invalidate because project 9530deaea206450d96772274828cca3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:42:05.452983 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-dcb5520d-bebe-44f0-ab62-26353912b99d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:05.453692 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-dcb5520d-bebe-44f0-ab62-26353912b99d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:05.456404 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 896/1781] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:42:05 2022] DELETE /identity/v3/projects/9530deaea206450d96772274828cca3b => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:05.458331 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7492fefa-9c21-4cd6-a8bd-89242c2a1b13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7122ca675422412182def8cc10480b39'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:05.467996 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb5520d-bebe-44f0-ab62-26353912b99d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:05.469800 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-dcb5520d-bebe-44f0-ab62-26353912b99d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:05.557703 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7492fefa-9c21-4cd6-a8bd-89242c2a1b13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7122ca675422412182def8cc10480b39'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:05.558457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7492fefa-9c21-4cd6-a8bd-89242c2a1b13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7122ca675422412182def8cc10480b39'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:05.559342 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7492fefa-9c21-4cd6-a8bd-89242c2a1b13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7122ca675422412182def8cc10480b39'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:05.569687 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7492fefa-9c21-4cd6-a8bd-89242c2a1b13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7122ca675422412182def8cc10480b39'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:05.569928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb5520d-bebe-44f0-ab62-26353912b99d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:05.575418 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106d5fd-e7ab-4962-a26f-f556e8f9945c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:05.576070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106d5fd-e7ab-4962-a26f-f556e8f9945c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:05.576353 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106d5fd-e7ab-4962-a26f-f556e8f9945c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:05.576655 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106d5fd-e7ab-4962-a26f-f556e8f9945c admin admin] PATH_INFO: `/v3/users/eb1f6270711447f08fe164404767f82d` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:05.578681 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106d5fd-e7ab-4962-a26f-f556e8f9945c admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb1f6270711447f08fe164404767f82d)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:05.579797 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106d5fd-e7ab-4962-a26f-f556e8f9945c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:05.585727 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:05.623264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 886/1782] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:42:05 2022] DELETE /identity/v3/users/7122ca675422412182def8cc10480b39 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:05.806963 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5106d5fd-e7ab-4962-a26f-f556e8f9945c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb1f6270711447f08fe164404767f82d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:05.824195 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5106d5fd-e7ab-4962-a26f-f556e8f9945c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb1f6270711447f08fe164404767f82d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:05.951054 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5106d5fd-e7ab-4962-a26f-f556e8f9945c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb1f6270711447f08fe164404767f82d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:05.951917 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5106d5fd-e7ab-4962-a26f-f556e8f9945c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb1f6270711447f08fe164404767f82d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:05.952712 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5106d5fd-e7ab-4962-a26f-f556e8f9945c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb1f6270711447f08fe164404767f82d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:06.001588 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 897/1783] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:42:05 2022] DELETE /identity/v3/users/eb1f6270711447f08fe164404767f82d => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:06.529126 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:06.546355 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492fefa-9c21-4cd6-a8bd-89242c2a1b13 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:06.548391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7492fefa-9c21-4cd6-a8bd-89242c2a1b13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:06.693451 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492fefa-9c21-4cd6-a8bd-89242c2a1b13 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:06.802523 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcf83c4-55f2-4cc7-875d-39e819413585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:06.803267 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf83c4-55f2-4cc7-875d-39e819413585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:06.803489 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf83c4-55f2-4cc7-875d-39e819413585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:06.803776 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf83c4-55f2-4cc7-875d-39e819413585 admin admin] PATH_INFO: `/v3/projects/4c497beedf944c469dd708dc69239b6f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:06.805551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcf83c4-55f2-4cc7-875d-39e819413585 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c497beedf944c469dd708dc69239b6f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:06.807524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcf83c4-55f2-4cc7-875d-39e819413585 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:06.918407 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106d5fd-e7ab-4962-a26f-f556e8f9945c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:06.919908 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5106d5fd-e7ab-4962-a26f-f556e8f9945c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:06.937064 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106d5fd-e7ab-4962-a26f-f556e8f9945c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:06.966516 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a284e-a8d2-4200-badb-134ba907a9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:06.967860 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a284e-a8d2-4200-badb-134ba907a9db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:06.972917 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a284e-a8d2-4200-badb-134ba907a9db admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:06.972917 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a284e-a8d2-4200-badb-134ba907a9db admin admin] PATH_INFO: `/v3/projects/c62ca1a705f34f12b1d7b7190c9cbf3a` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:06.975759 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a284e-a8d2-4200-badb-134ba907a9db admin admin] RBAC: Authorizing `identity:delete_project(project_id=c62ca1a705f34f12b1d7b7190c9cbf3a)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:06.978300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a284e-a8d2-4200-badb-134ba907a9db admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:06.978939 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3dcf83c4-55f2-4cc7-875d-39e819413585 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c497beedf944c469dd708dc69239b6f', 'request_id': 'req-3dcf83c4-55f2-4cc7-875d-39e819413585'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:06.986510 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3dcf83c4-55f2-4cc7-875d-39e819413585 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c497beedf944c469dd708dc69239b6f', 'request_id': 'req-3dcf83c4-55f2-4cc7-875d-39e819413585'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:08.115959 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3dcf83c4-55f2-4cc7-875d-39e819413585 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c497beedf944c469dd708dc69239b6f', 'request_id': 'req-3dcf83c4-55f2-4cc7-875d-39e819413585'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:08.129375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3dcf83c4-55f2-4cc7-875d-39e819413585 admin admin] The token cache is being invalidate because project 4c497beedf944c469dd708dc69239b6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:42:08.130800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3dcf83c4-55f2-4cc7-875d-39e819413585 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:08.131675 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3dcf83c4-55f2-4cc7-875d-39e819413585 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:08.133480 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 887/1784] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:42:06 2022] DELETE /identity/v3/projects/4c497beedf944c469dd708dc69239b6f => generated 0 bytes in 1587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:08.136116 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-239a284e-a8d2-4200-badb-134ba907a9db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c62ca1a705f34f12b1d7b7190c9cbf3a', 'request_id': 'req-239a284e-a8d2-4200-badb-134ba907a9db'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:08.137085 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcf83c4-55f2-4cc7-875d-39e819413585 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:08.138623 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3dcf83c4-55f2-4cc7-875d-39e819413585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:08.141838 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-239a284e-a8d2-4200-badb-134ba907a9db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c62ca1a705f34f12b1d7b7190c9cbf3a', 'request_id': 'req-239a284e-a8d2-4200-badb-134ba907a9db'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:08.207346 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcf83c4-55f2-4cc7-875d-39e819413585 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:08.244943 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-239a284e-a8d2-4200-badb-134ba907a9db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c62ca1a705f34f12b1d7b7190c9cbf3a', 'request_id': 'req-239a284e-a8d2-4200-badb-134ba907a9db'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:08.268262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-239a284e-a8d2-4200-badb-134ba907a9db admin admin] The token cache is being invalidate because project c62ca1a705f34f12b1d7b7190c9cbf3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:42:08.269018 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-239a284e-a8d2-4200-badb-134ba907a9db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:08.270706 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-239a284e-a8d2-4200-badb-134ba907a9db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:08.273442 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 898/1785] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:42:06 2022] DELETE /identity/v3/projects/c62ca1a705f34f12b1d7b7190c9cbf3a => generated 0 bytes in 1355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:42:08.279035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43bc7b-0000-4a91-a3e2-7d9bcd94daa0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:08.279779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43bc7b-0000-4a91-a3e2-7d9bcd94daa0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:08.279779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43bc7b-0000-4a91-a3e2-7d9bcd94daa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:08.304642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26939303-b585-4d35-b704-a20cef5b389a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:08.305294 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26939303-b585-4d35-b704-a20cef5b389a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:08.305572 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26939303-b585-4d35-b704-a20cef5b389a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:08.305849 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26939303-b585-4d35-b704-a20cef5b389a admin admin] PATH_INFO: `/v3/users/fb689d487add49b193f0a7845efca4fc` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:08.307161 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26939303-b585-4d35-b704-a20cef5b389a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb689d487add49b193f0a7845efca4fc)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:08.309174 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26939303-b585-4d35-b704-a20cef5b389a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:08.325388 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-fa43bc7b-0000-4a91-a3e2-7d9bcd94daa0 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:08.338531 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-fa43bc7b-0000-4a91-a3e2-7d9bcd94daa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:08.359704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 899/1786] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:08.367937 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd970e-de65-4ec7-80db-bdfa35e8f2b3 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:08.368225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd970e-de65-4ec7-80db-bdfa35e8f2b3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:08.368496 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd970e-de65-4ec7-80db-bdfa35e8f2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:08.395032 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-67dd970e-de65-4ec7-80db-bdfa35e8f2b3 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:08.396315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-67dd970e-de65-4ec7-80db-bdfa35e8f2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:08.404621 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 900/1787] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:42:08.408971 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd970e-de65-4ec7-80db-bdfa35e8f2b3 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:08.430240 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd970e-de65-4ec7-80db-bdfa35e8f2b3 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:08.435866 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f4805-ea42-47d5-8d10-936ac696d8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:08.436672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f4805-ea42-47d5-8d10-936ac696d8df admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:08.436747 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f4805-ea42-47d5-8d10-936ac696d8df admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:08.437039 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f4805-ea42-47d5-8d10-936ac696d8df admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:08.437661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f4805-ea42-47d5-8d10-936ac696d8df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:42:08.437977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f4805-ea42-47d5-8d10-936ac696d8df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:08.439634 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f4805-ea42-47d5-8d10-936ac696d8df admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:08.457278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 901/1788] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:42:08.458392 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f4805-ea42-47d5-8d10-936ac696d8df admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:08.467204 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:08.490587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f4805-ea42-47d5-8d10-936ac696d8df admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:08.498368 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d0d28-dc04-4899-83ff-425fd1fe94d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:08.498999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d0d28-dc04-4899-83ff-425fd1fe94d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:08.504016 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d0d28-dc04-4899-83ff-425fd1fe94d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:08.504016 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d0d28-dc04-4899-83ff-425fd1fe94d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:08.504016 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824d0d28-dc04-4899-83ff-425fd1fe94d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:08.504016 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824d0d28-dc04-4899-83ff-425fd1fe94d0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:08.578358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-26939303-b585-4d35-b704-a20cef5b389a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb689d487add49b193f0a7845efca4fc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:08.595525 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 902/1789] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:42:08 2022] POST /identity/v3/projects => generated 383 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:42:08.597905 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d0d28-dc04-4899-83ff-425fd1fe94d0 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:08.663785 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d0d28-dc04-4899-83ff-425fd1fe94d0 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:08.673124 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c3479-658a-446e-b352-5d48a01de640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:08.674417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c3479-658a-446e-b352-5d48a01de640 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:08.674417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c3479-658a-446e-b352-5d48a01de640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:08.674417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c3479-658a-446e-b352-5d48a01de640 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:08.674977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c3479-658a-446e-b352-5d48a01de640 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:08.677665 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c3479-658a-446e-b352-5d48a01de640 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:08.694583 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-26939303-b585-4d35-b704-a20cef5b389a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb689d487add49b193f0a7845efca4fc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:08.695594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-26939303-b585-4d35-b704-a20cef5b389a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb689d487add49b193f0a7845efca4fc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:08.705977 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-26939303-b585-4d35-b704-a20cef5b389a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb689d487add49b193f0a7845efca4fc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:08.709794 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-26939303-b585-4d35-b704-a20cef5b389a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb689d487add49b193f0a7845efca4fc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:08.733856 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 888/1790] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:42:08 2022] DELETE /identity/v3/users/fb689d487add49b193f0a7845efca4fc => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:42:08.735569 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5538651-0600-4863-a194-994b8f2299cc None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:08.739639 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5538651-0600-4863-a194-994b8f2299cc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:08.739639 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5538651-0600-4863-a194-994b8f2299cc None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:08.739639 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 889/1791] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 10 12:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:42:08.794604 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 903/1792] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:42:08 2022] POST /identity/v3/users => generated 292 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:42:08.799494 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5538651-0600-4863-a194-994b8f2299cc None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:08.802512 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a5538651-0600-4863-a194-994b8f2299cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:08.806750 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5989e7d-a5d7-42df-967f-a133461b8774 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:08.814093 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5989e7d-a5d7-42df-967f-a133461b8774 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:08.815754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5989e7d-a5d7-42df-967f-a133461b8774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:08.881717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f5989e7d-a5d7-42df-967f-a133461b8774 None None] MFA Rules not processed for user `eb3f6deffb54467f8b923956ed2a6acf`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:08.887653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f5989e7d-a5d7-42df-967f-a133461b8774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:08.969560 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5538651-0600-4863-a194-994b8f2299cc None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:08.977324 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb7cd3-cf21-4807-a396-ba63a53e7148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:08.978109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb7cd3-cf21-4807-a396-ba63a53e7148 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:08.978487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb7cd3-cf21-4807-a396-ba63a53e7148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:08.979494 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb7cd3-cf21-4807-a396-ba63a53e7148 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:08.980461 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bb7cd3-cf21-4807-a396-ba63a53e7148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:08.987337 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 904/1793] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 10 12:42:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:08.987492 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bb7cd3-cf21-4807-a396-ba63a53e7148 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:08.998414 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 890/1794] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:09.002261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5989e7d-a5d7-42df-967f-a133461b8774 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:09.017958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5989e7d-a5d7-42df-967f-a133461b8774 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:09.025429 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49981524-2962-448e-9118-f78047bfed1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:09.026007 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49981524-2962-448e-9118-f78047bfed1a admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:09.026299 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49981524-2962-448e-9118-f78047bfed1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:09.026786 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49981524-2962-448e-9118-f78047bfed1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:09.027622 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49981524-2962-448e-9118-f78047bfed1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:09.030324 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49981524-2962-448e-9118-f78047bfed1a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:09.048961 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 905/1795] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:09.050280 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb7cd3-cf21-4807-a396-ba63a53e7148 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:09.063363 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb7cd3-cf21-4807-a396-ba63a53e7148 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:09.070480 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010562d-6547-4b17-b3cf-62675904a630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:09.071292 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010562d-6547-4b17-b3cf-62675904a630 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:09.071721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010562d-6547-4b17-b3cf-62675904a630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:09.072011 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010562d-6547-4b17-b3cf-62675904a630 admin admin] PATH_INFO: `/v3/projects/79b917e5c2ff48f3968b6c55e54a69b2/users/9109f1122aa74e0e9c392d2ceb19c770/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:09.092275 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010562d-6547-4b17-b3cf-62675904a630 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79b917e5c2ff48f3968b6c55e54a69b2, user_id=9109f1122aa74e0e9c392d2ceb19c770, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:09.094912 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010562d-6547-4b17-b3cf-62675904a630 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:09.195991 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 891/1796] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:42:09 2022] PUT /identity/v3/projects/79b917e5c2ff48f3968b6c55e54a69b2/users/9109f1122aa74e0e9c392d2ceb19c770/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:09.200328 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a94a27-cc41-4099-81d0-403aa8f03b26 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:09.200724 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a94a27-cc41-4099-81d0-403aa8f03b26 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:09.200823 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a94a27-cc41-4099-81d0-403aa8f03b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:09.270528 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d8a94a27-cc41-4099-81d0-403aa8f03b26 None None] MFA Rules not processed for user `9109f1122aa74e0e9c392d2ceb19c770`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:09.271802 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d8a94a27-cc41-4099-81d0-403aa8f03b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:09.386038 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 906/1797] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:09.387542 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010562d-6547-4b17-b3cf-62675904a630 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:09.461976 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010562d-6547-4b17-b3cf-62675904a630 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:09.473452 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24add503-e797-456d-93a9-f4d7a29dd13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:09.474171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24add503-e797-456d-93a9-f4d7a29dd13a admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:09.474641 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24add503-e797-456d-93a9-f4d7a29dd13a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:09.475094 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24add503-e797-456d-93a9-f4d7a29dd13a admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:09.475848 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24add503-e797-456d-93a9-f4d7a29dd13a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:09.479223 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24add503-e797-456d-93a9-f4d7a29dd13a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:09.593218 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 892/1798] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:42:09 2022] POST /identity/v3/projects => generated 383 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:42:09.596985 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a94a27-cc41-4099-81d0-403aa8f03b26 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:09.607842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a94a27-cc41-4099-81d0-403aa8f03b26 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:09.666946 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503fd4d5-a3a8-4903-af53-2bcdcd6a34e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:09.667685 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fd4d5-a3a8-4903-af53-2bcdcd6a34e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:09.668565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fd4d5-a3a8-4903-af53-2bcdcd6a34e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:09.668565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fd4d5-a3a8-4903-af53-2bcdcd6a34e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:09.669069 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503fd4d5-a3a8-4903-af53-2bcdcd6a34e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:09.670986 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503fd4d5-a3a8-4903-af53-2bcdcd6a34e4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:09.737233 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24add503-e797-456d-93a9-f4d7a29dd13a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:09.740738 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-24add503-e797-456d-93a9-f4d7a29dd13a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:09.800834 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 907/1799] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:42:09 2022] POST /identity/v3/users => generated 298 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:42:09.800834 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503fd4d5-a3a8-4903-af53-2bcdcd6a34e4 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:09.816868 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503fd4d5-a3a8-4903-af53-2bcdcd6a34e4 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:09.822547 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:09.833602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930aaccb-76f1-4823-8a2a-89075a8f5986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:09.834250 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930aaccb-76f1-4823-8a2a-89075a8f5986 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:09.834844 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930aaccb-76f1-4823-8a2a-89075a8f5986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:09.835132 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930aaccb-76f1-4823-8a2a-89075a8f5986 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:09.835828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930aaccb-76f1-4823-8a2a-89075a8f5986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:09.837411 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930aaccb-76f1-4823-8a2a-89075a8f5986 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:09.851139 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24add503-e797-456d-93a9-f4d7a29dd13a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:09.858071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44979547-c64d-4843-989f-093c1b6ddc51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:09.858849 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44979547-c64d-4843-989f-093c1b6ddc51 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:09.859345 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44979547-c64d-4843-989f-093c1b6ddc51 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:09.859623 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44979547-c64d-4843-989f-093c1b6ddc51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:09.874507 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 908/1800] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:42:09.874641 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44979547-c64d-4843-989f-093c1b6ddc51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:09.876511 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44979547-c64d-4843-989f-093c1b6ddc51 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:09.876990 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930aaccb-76f1-4823-8a2a-89075a8f5986 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:09.881428 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-930aaccb-76f1-4823-8a2a-89075a8f5986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:09.890619 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 893/1801] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:42:09.891501 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44979547-c64d-4843-989f-093c1b6ddc51 service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:09.902728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44979547-c64d-4843-989f-093c1b6ddc51 service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:09.908746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06436e8a-f6b2-4098-b6dc-308e92638729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:09.909418 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06436e8a-f6b2-4098-b6dc-308e92638729 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:09.909739 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06436e8a-f6b2-4098-b6dc-308e92638729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:09.910194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06436e8a-f6b2-4098-b6dc-308e92638729 admin admin] PATH_INFO: `/v3/projects/eb0d794df07d424091a0c59a86c4adca/users/458998dee69e47848785710e4b03550b/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:09.932291 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06436e8a-f6b2-4098-b6dc-308e92638729 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb0d794df07d424091a0c59a86c4adca, user_id=458998dee69e47848785710e4b03550b, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:09.932291 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06436e8a-f6b2-4098-b6dc-308e92638729 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:09.936903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930aaccb-76f1-4823-8a2a-89075a8f5986 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:09.954855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba291f-0aa4-4b43-9f8a-58a60e448e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:09.955574 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba291f-0aa4-4b43-9f8a-58a60e448e65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:09.955862 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba291f-0aa4-4b43-9f8a-58a60e448e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:09.956144 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba291f-0aa4-4b43-9f8a-58a60e448e65 admin admin] PATH_INFO: `/v3/projects/734c44a7ef654afb8a2ebd83f2f67bf3` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:09.957812 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba291f-0aa4-4b43-9f8a-58a60e448e65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=734c44a7ef654afb8a2ebd83f2f67bf3)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:09.959757 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba291f-0aa4-4b43-9f8a-58a60e448e65 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:10.052389 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 894/1802] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:42:09 2022] PUT /identity/v3/projects/eb0d794df07d424091a0c59a86c4adca/users/458998dee69e47848785710e4b03550b/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:10.053476 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06436e8a-f6b2-4098-b6dc-308e92638729 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:10.157208 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f8ba291f-0aa4-4b43-9f8a-58a60e448e65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '734c44a7ef654afb8a2ebd83f2f67bf3', 'request_id': 'req-f8ba291f-0aa4-4b43-9f8a-58a60e448e65'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:10.170272 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f8ba291f-0aa4-4b43-9f8a-58a60e448e65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '734c44a7ef654afb8a2ebd83f2f67bf3', 'request_id': 'req-f8ba291f-0aa4-4b43-9f8a-58a60e448e65'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:10.179458 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06436e8a-f6b2-4098-b6dc-308e92638729 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:10.267941 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc7b69-0784-49af-ad6e-74e0a473902c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:10.268617 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc7b69-0784-49af-ad6e-74e0a473902c admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:10.271720 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f8ba291f-0aa4-4b43-9f8a-58a60e448e65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '734c44a7ef654afb8a2ebd83f2f67bf3', 'request_id': 'req-f8ba291f-0aa4-4b43-9f8a-58a60e448e65'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:10.280546 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc7b69-0784-49af-ad6e-74e0a473902c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:10.280977 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc7b69-0784-49af-ad6e-74e0a473902c admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:10.283078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc7b69-0784-49af-ad6e-74e0a473902c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:10.287946 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc7b69-0784-49af-ad6e-74e0a473902c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:10.293192 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f8ba291f-0aa4-4b43-9f8a-58a60e448e65 admin admin] The token cache is being invalidate because project 734c44a7ef654afb8a2ebd83f2f67bf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:42:10.295799 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f8ba291f-0aa4-4b43-9f8a-58a60e448e65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:10.299256 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f8ba291f-0aa4-4b43-9f8a-58a60e448e65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:10.303544 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 909/1803] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:42:09 2022] DELETE /identity/v3/projects/734c44a7ef654afb8a2ebd83f2f67bf3 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:10.309339 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b392c1-ab32-4abc-9c43-4845a3a9d451 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:10.309782 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b392c1-ab32-4abc-9c43-4845a3a9d451 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:10.310296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b392c1-ab32-4abc-9c43-4845a3a9d451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:10.315045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 895/1804] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:10.320107 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc7b69-0784-49af-ad6e-74e0a473902c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:10.323629 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-41bc7b69-0784-49af-ad6e-74e0a473902c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:10.355822 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc7b69-0784-49af-ad6e-74e0a473902c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:10.357836 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-05b392c1-ab32-4abc-9c43-4845a3a9d451 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:10.359561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-05b392c1-ab32-4abc-9c43-4845a3a9d451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:10.365241 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00851ddc-b0a0-4548-b620-303afd15f5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:10.366023 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00851ddc-b0a0-4548-b620-303afd15f5dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:10.366700 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00851ddc-b0a0-4548-b620-303afd15f5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:10.367254 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00851ddc-b0a0-4548-b620-303afd15f5dd admin admin] PATH_INFO: `/v3/projects/eb0d794df07d424091a0c59a86c4adca/users/458998dee69e47848785710e4b03550b/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:10.370190 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00851ddc-b0a0-4548-b620-303afd15f5dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb0d794df07d424091a0c59a86c4adca, user_id=458998dee69e47848785710e4b03550b, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:10.373192 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00851ddc-b0a0-4548-b620-303afd15f5dd admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:10.381920 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 910/1805] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:42:10.391888 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 896/1806] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:42:10 2022] PUT /identity/v3/projects/eb0d794df07d424091a0c59a86c4adca/users/458998dee69e47848785710e4b03550b/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:10.397866 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45170744-5ff1-4f63-bf6b-2f11f172c514 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:10.398024 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45170744-5ff1-4f63-bf6b-2f11f172c514 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:10.398354 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45170744-5ff1-4f63-bf6b-2f11f172c514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:10.399565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd29e44-689b-4051-a56c-f6b55e24e606 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:10.399928 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd29e44-689b-4051-a56c-f6b55e24e606 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:10.400702 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd29e44-689b-4051-a56c-f6b55e24e606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:10.435219 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-45170744-5ff1-4f63-bf6b-2f11f172c514 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:10.436746 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-45170744-5ff1-4f63-bf6b-2f11f172c514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:10.458702 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-2cd29e44-689b-4051-a56c-f6b55e24e606 None None] MFA Rules not processed for user `458998dee69e47848785710e4b03550b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:10.460162 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2cd29e44-689b-4051-a56c-f6b55e24e606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:10.531945 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 911/1807] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:10.536413 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45170744-5ff1-4f63-bf6b-2f11f172c514 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:10.567581 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45170744-5ff1-4f63-bf6b-2f11f172c514 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:10.578701 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ebd55-456c-4db0-91f6-c409dc3a488a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:10.584079 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ebd55-456c-4db0-91f6-c409dc3a488a admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:10.584079 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ebd55-456c-4db0-91f6-c409dc3a488a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:10.584079 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ebd55-456c-4db0-91f6-c409dc3a488a admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:10.584079 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ebd55-456c-4db0-91f6-c409dc3a488a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:42:10.584079 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ebd55-456c-4db0-91f6-c409dc3a488a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:10.584079 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ebd55-456c-4db0-91f6-c409dc3a488a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:10.596677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 912/1808] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:42:10.601257 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 897/1809] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:10 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:10.604370 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ebd55-456c-4db0-91f6-c409dc3a488a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:10.616107 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ebd55-456c-4db0-91f6-c409dc3a488a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:10.616875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd29e44-689b-4051-a56c-f6b55e24e606 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:10.618787 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2cd29e44-689b-4051-a56c-f6b55e24e606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:10.625231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ef5e1-282a-478f-8027-126556c7a544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:10.625977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ef5e1-282a-478f-8027-126556c7a544 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:10.628947 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ef5e1-282a-478f-8027-126556c7a544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:10.628947 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ef5e1-282a-478f-8027-126556c7a544 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:10.628947 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ef5e1-282a-478f-8027-126556c7a544 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:10.631791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ef5e1-282a-478f-8027-126556c7a544 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:10.719012 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd29e44-689b-4051-a56c-f6b55e24e606 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:10.724107 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5101e9bf-79ec-48a8-86e8-b06e73dc7850 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:10.725053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101e9bf-79ec-48a8-86e8-b06e73dc7850 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:10.725438 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101e9bf-79ec-48a8-86e8-b06e73dc7850 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:10.726094 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101e9bf-79ec-48a8-86e8-b06e73dc7850 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:10.742662 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5101e9bf-79ec-48a8-86e8-b06e73dc7850 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:10.744848 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5101e9bf-79ec-48a8-86e8-b06e73dc7850 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:10.758757 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 898/1810] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:10 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:42:10.825551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 913/1811] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:42:10 2022] POST /identity/v3/projects => generated 383 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:42:10.830433 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5101e9bf-79ec-48a8-86e8-b06e73dc7850 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:10.846034 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5101e9bf-79ec-48a8-86e8-b06e73dc7850 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:10.902104 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ef5e1-282a-478f-8027-126556c7a544 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:10.907581 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-533ef5e1-282a-478f-8027-126556c7a544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:10.923891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee008a86-2ffd-4cd3-9777-84e01ad8daef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:10.924535 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee008a86-2ffd-4cd3-9777-84e01ad8daef admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:10.924825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee008a86-2ffd-4cd3-9777-84e01ad8daef admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:10.925102 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee008a86-2ffd-4cd3-9777-84e01ad8daef admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:10.925750 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee008a86-2ffd-4cd3-9777-84e01ad8daef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:10.928609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee008a86-2ffd-4cd3-9777-84e01ad8daef admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:10.990838 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ef5e1-282a-478f-8027-126556c7a544 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:10.991455 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:11.015299 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 899/1812] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:42:10 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:42:11.020785 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee008a86-2ffd-4cd3-9777-84e01ad8daef admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:11.020928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefcd60-1da1-4995-a6ca-db907c98c7a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:11.021224 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefcd60-1da1-4995-a6ca-db907c98c7a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:11.021526 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefcd60-1da1-4995-a6ca-db907c98c7a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:11.024859 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefcd60-1da1-4995-a6ca-db907c98c7a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:11.028439 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee008a86-2ffd-4cd3-9777-84e01ad8daef admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:11.030880 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bdefcd60-1da1-4995-a6ca-db907c98c7a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:11.034785 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a855fe-565c-4129-87f0-03b68c77c38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:11.035753 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a855fe-565c-4129-87f0-03b68c77c38d admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:11.036139 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a855fe-565c-4129-87f0-03b68c77c38d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:11.036568 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a855fe-565c-4129-87f0-03b68c77c38d admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:11.037509 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a855fe-565c-4129-87f0-03b68c77c38d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:11.040251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a855fe-565c-4129-87f0-03b68c77c38d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:11.053942 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 900/1813] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:11.058829 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a855fe-565c-4129-87f0-03b68c77c38d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:11.075271 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a855fe-565c-4129-87f0-03b68c77c38d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:11.082698 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4c1a2-9056-46c9-bc31-982f4b2ad671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:11.083575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4c1a2-9056-46c9-bc31-982f4b2ad671 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:11.083963 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4c1a2-9056-46c9-bc31-982f4b2ad671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:11.084337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4c1a2-9056-46c9-bc31-982f4b2ad671 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:11.085085 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde4c1a2-9056-46c9-bc31-982f4b2ad671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:11.086930 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde4c1a2-9056-46c9-bc31-982f4b2ad671 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:11.103200 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 901/1814] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:42:11.103410 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdefcd60-1da1-4995-a6ca-db907c98c7a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:11.104032 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdefcd60-1da1-4995-a6ca-db907c98c7a8 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:11.104679 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4c1a2-9056-46c9-bc31-982f4b2ad671 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:11.116649 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 914/1815] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:42:11.118053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4c1a2-9056-46c9-bc31-982f4b2ad671 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:11.130537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1461d35-0cd6-4066-beb2-97c87b6c2dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:11.130537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1461d35-0cd6-4066-beb2-97c87b6c2dc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:11.130537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1461d35-0cd6-4066-beb2-97c87b6c2dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:11.130537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1461d35-0cd6-4066-beb2-97c87b6c2dc7 admin admin] PATH_INFO: `/v3/projects/7ffd5a842fa449a5b1a9375220bfede8/users/acbb2497f2eb4ac99dc8b235e7b6ea3c/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:11.143809 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1461d35-0cd6-4066-beb2-97c87b6c2dc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ffd5a842fa449a5b1a9375220bfede8, user_id=acbb2497f2eb4ac99dc8b235e7b6ea3c, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:11.147086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1461d35-0cd6-4066-beb2-97c87b6c2dc7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:11.238837 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 902/1816] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:42:11 2022] PUT /identity/v3/projects/7ffd5a842fa449a5b1a9375220bfede8/users/acbb2497f2eb4ac99dc8b235e7b6ea3c/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:11.245036 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7482d-4094-4052-a582-fde878c6d453 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:11.245354 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7482d-4094-4052-a582-fde878c6d453 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:11.245639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7482d-4094-4052-a582-fde878c6d453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:11.279924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3fb7482d-4094-4052-a582-fde878c6d453 None None] MFA Rules not processed for user `acbb2497f2eb4ac99dc8b235e7b6ea3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:11.281188 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3fb7482d-4094-4052-a582-fde878c6d453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:11.349409 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 915/1817] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:11.353777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1461d35-0cd6-4066-beb2-97c87b6c2dc7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:11.420884 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1461d35-0cd6-4066-beb2-97c87b6c2dc7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:11.431010 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb096d-64b0-4d09-aafb-34f862d8a2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:11.431903 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb096d-64b0-4d09-aafb-34f862d8a2f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:11.438795 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb096d-64b0-4d09-aafb-34f862d8a2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:11.439262 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb096d-64b0-4d09-aafb-34f862d8a2f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:11.440070 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb096d-64b0-4d09-aafb-34f862d8a2f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:11.442766 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb096d-64b0-4d09-aafb-34f862d8a2f2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:11.591530 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 903/1818] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:42:11 2022] POST /identity/v3/projects => generated 383 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:42:11.595666 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7482d-4094-4052-a582-fde878c6d453 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:11.606592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7482d-4094-4052-a582-fde878c6d453 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:11.680353 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d21179-3bae-4b68-8932-3da2764f2598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:11.680973 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d21179-3bae-4b68-8932-3da2764f2598 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:11.681259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d21179-3bae-4b68-8932-3da2764f2598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:11.681536 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d21179-3bae-4b68-8932-3da2764f2598 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:11.682163 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d21179-3bae-4b68-8932-3da2764f2598 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:11.684128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d21179-3bae-4b68-8932-3da2764f2598 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:11.810857 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 916/1819] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:42:11 2022] POST /identity/v3/users => generated 298 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:42:11.812285 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb096d-64b0-4d09-aafb-34f862d8a2f2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:11.823455 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb096d-64b0-4d09-aafb-34f862d8a2f2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:11.828507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac1c6a-ca8b-4323-a535-af67da791711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:11.829257 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac1c6a-ca8b-4323-a535-af67da791711 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:11.829540 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac1c6a-ca8b-4323-a535-af67da791711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:11.829812 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac1c6a-ca8b-4323-a535-af67da791711 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:11.830443 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac1c6a-ca8b-4323-a535-af67da791711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:11.832143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac1c6a-ca8b-4323-a535-af67da791711 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:11.844518 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 904/1820] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:42:11.854274 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d21179-3bae-4b68-8932-3da2764f2598 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:11.865320 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d21179-3bae-4b68-8932-3da2764f2598 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:11.871632 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b78be-5634-456c-845d-fe299f2e54e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:11.872471 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b78be-5634-456c-845d-fe299f2e54e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:11.872589 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b78be-5634-456c-845d-fe299f2e54e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:11.872849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b78be-5634-456c-845d-fe299f2e54e0 admin admin] PATH_INFO: `/v3/projects/461df23abd9b454b94e4f6f5bb542b39/users/57b031573a134d5ab5efd6290ea9a110/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:11.907072 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b78be-5634-456c-845d-fe299f2e54e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=461df23abd9b454b94e4f6f5bb542b39, user_id=57b031573a134d5ab5efd6290ea9a110, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:11.919595 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b78be-5634-456c-845d-fe299f2e54e0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:11.985751 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 917/1821] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:42:11 2022] PUT /identity/v3/projects/461df23abd9b454b94e4f6f5bb542b39/users/57b031573a134d5ab5efd6290ea9a110/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:11.991406 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac1c6a-ca8b-4323-a535-af67da791711 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:12.007133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac1c6a-ca8b-4323-a535-af67da791711 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:12.134723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74725bfb-51ac-4339-9d1e-28fa4e10c77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:12.135427 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74725bfb-51ac-4339-9d1e-28fa4e10c77c admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:12.135893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74725bfb-51ac-4339-9d1e-28fa4e10c77c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:12.136177 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74725bfb-51ac-4339-9d1e-28fa4e10c77c admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:12.142162 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74725bfb-51ac-4339-9d1e-28fa4e10c77c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:12.142162 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74725bfb-51ac-4339-9d1e-28fa4e10c77c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:12.153386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 905/1822] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:12.158160 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b78be-5634-456c-845d-fe299f2e54e0 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:12.170029 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b78be-5634-456c-845d-fe299f2e54e0 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:12.175558 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050acf5-e7bd-45ef-925f-aa95ab863d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:12.176977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050acf5-e7bd-45ef-925f-aa95ab863d22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:12.177390 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050acf5-e7bd-45ef-925f-aa95ab863d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:12.177766 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050acf5-e7bd-45ef-925f-aa95ab863d22 admin admin] PATH_INFO: `/v3/projects/461df23abd9b454b94e4f6f5bb542b39/users/57b031573a134d5ab5efd6290ea9a110/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:12.179934 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3050acf5-e7bd-45ef-925f-aa95ab863d22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=461df23abd9b454b94e4f6f5bb542b39, user_id=57b031573a134d5ab5efd6290ea9a110, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:12.183192 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3050acf5-e7bd-45ef-925f-aa95ab863d22 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:12.196103 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 918/1823] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:42:12 2022] PUT /identity/v3/projects/461df23abd9b454b94e4f6f5bb542b39/users/57b031573a134d5ab5efd6290ea9a110/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:12.206587 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6f8ad-703c-4582-8685-d5f4c0368da3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:12.207198 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6f8ad-703c-4582-8685-d5f4c0368da3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:12.208805 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6f8ad-703c-4582-8685-d5f4c0368da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:12.238738 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f5a6f8ad-703c-4582-8685-d5f4c0368da3 None None] MFA Rules not processed for user `57b031573a134d5ab5efd6290ea9a110`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:12.240662 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f5a6f8ad-703c-4582-8685-d5f4c0368da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:12.328924 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 906/1824] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:12 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:12.341373 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050acf5-e7bd-45ef-925f-aa95ab863d22 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:12.407410 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050acf5-e7bd-45ef-925f-aa95ab863d22 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:12.412004 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c38c3a-f687-4486-bdcf-1a2086f97c4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:12.412618 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c38c3a-f687-4486-bdcf-1a2086f97c4f service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:12.412917 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c38c3a-f687-4486-bdcf-1a2086f97c4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:12.413176 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c38c3a-f687-4486-bdcf-1a2086f97c4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:12.422809 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c38c3a-f687-4486-bdcf-1a2086f97c4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:12.424282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c38c3a-f687-4486-bdcf-1a2086f97c4f service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:12.435579 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 919/1825] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:42:12.735034 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6f8ad-703c-4582-8685-d5f4c0368da3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:12.746026 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6f8ad-703c-4582-8685-d5f4c0368da3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:12.749703 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13754712-e581-4553-badb-99c76de24173 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:12.750312 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13754712-e581-4553-badb-99c76de24173 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:12.750589 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13754712-e581-4553-badb-99c76de24173 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:12.750859 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13754712-e581-4553-badb-99c76de24173 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:12.809195 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13754712-e581-4553-badb-99c76de24173 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:12.810704 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13754712-e581-4553-badb-99c76de24173 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:12.820174 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 907/1826] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:42:12.992756 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:13.485277 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c38c3a-f687-4486-bdcf-1a2086f97c4f service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:13.552693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c38c3a-f687-4486-bdcf-1a2086f97c4f service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:13.557497 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94635d79-85b5-434a-8f22-8140d6374636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:13.564853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94635d79-85b5-434a-8f22-8140d6374636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:13.564853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94635d79-85b5-434a-8f22-8140d6374636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:13.564853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94635d79-85b5-434a-8f22-8140d6374636 admin admin] PATH_INFO: `/v3/users/acbb2497f2eb4ac99dc8b235e7b6ea3c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:13.564853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94635d79-85b5-434a-8f22-8140d6374636 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acbb2497f2eb4ac99dc8b235e7b6ea3c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:13.564853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94635d79-85b5-434a-8f22-8140d6374636 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:13.809558 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-94635d79-85b5-434a-8f22-8140d6374636 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acbb2497f2eb4ac99dc8b235e7b6ea3c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:13.821006 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-94635d79-85b5-434a-8f22-8140d6374636 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acbb2497f2eb4ac99dc8b235e7b6ea3c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:13.904988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-94635d79-85b5-434a-8f22-8140d6374636 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acbb2497f2eb4ac99dc8b235e7b6ea3c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:13.905831 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-94635d79-85b5-434a-8f22-8140d6374636 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acbb2497f2eb4ac99dc8b235e7b6ea3c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:13.906536 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-94635d79-85b5-434a-8f22-8140d6374636 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acbb2497f2eb4ac99dc8b235e7b6ea3c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:13.933435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 920/1827] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:42:13 2022] DELETE /identity/v3/users/acbb2497f2eb4ac99dc8b235e7b6ea3c => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:13.938762 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27a6c0-397c-40e8-b33b-e98005cd7f44 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:13.939532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27a6c0-397c-40e8-b33b-e98005cd7f44 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:13.939532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27a6c0-397c-40e8-b33b-e98005cd7f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:13.961010 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5d27a6c0-397c-40e8-b33b-e98005cd7f44 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:13.962237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5d27a6c0-397c-40e8-b33b-e98005cd7f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:14.037860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 908/1828] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:42:14.047842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94635d79-85b5-434a-8f22-8140d6374636 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:14.050041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-94635d79-85b5-434a-8f22-8140d6374636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:14.133634 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94635d79-85b5-434a-8f22-8140d6374636 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:14.137988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee5f3e-b622-484f-82b9-b79000895edf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:14.138757 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee5f3e-b622-484f-82b9-b79000895edf service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:14.139039 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee5f3e-b622-484f-82b9-b79000895edf service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:14.139346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee5f3e-b622-484f-82b9-b79000895edf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:14.154162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ee5f3e-b622-484f-82b9-b79000895edf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:14.156177 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ee5f3e-b622-484f-82b9-b79000895edf service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:14.172179 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 921/1829] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:42:14.224500 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:14.446327 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27a6c0-397c-40e8-b33b-e98005cd7f44 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:14.449731 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5d27a6c0-397c-40e8-b33b-e98005cd7f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:14.473719 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27a6c0-397c-40e8-b33b-e98005cd7f44 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:14.497548 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b0ffc-9bb7-45b7-84f8-288c3e6bd76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:14.498211 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b0ffc-9bb7-45b7-84f8-288c3e6bd76d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:14.498501 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b0ffc-9bb7-45b7-84f8-288c3e6bd76d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:14.498774 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b0ffc-9bb7-45b7-84f8-288c3e6bd76d admin admin] PATH_INFO: `/v3/users/9109f1122aa74e0e9c392d2ceb19c770` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:14.500079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5b0ffc-9bb7-45b7-84f8-288c3e6bd76d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9109f1122aa74e0e9c392d2ceb19c770)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:14.502188 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5b0ffc-9bb7-45b7-84f8-288c3e6bd76d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:14.703220 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8f5b0ffc-9bb7-45b7-84f8-288c3e6bd76d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9109f1122aa74e0e9c392d2ceb19c770'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:14.806338 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8f5b0ffc-9bb7-45b7-84f8-288c3e6bd76d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9109f1122aa74e0e9c392d2ceb19c770'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:14.807054 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8f5b0ffc-9bb7-45b7-84f8-288c3e6bd76d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9109f1122aa74e0e9c392d2ceb19c770'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:14.807762 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8f5b0ffc-9bb7-45b7-84f8-288c3e6bd76d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9109f1122aa74e0e9c392d2ceb19c770'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:14.815484 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8f5b0ffc-9bb7-45b7-84f8-288c3e6bd76d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9109f1122aa74e0e9c392d2ceb19c770'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:14.838559 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 909/1830] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:42:14 2022] DELETE /identity/v3/users/9109f1122aa74e0e9c392d2ceb19c770 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:14.840196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1f1ea-d5e6-4c15-bfd3-79028bcc1e8b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:14.840667 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1f1ea-d5e6-4c15-bfd3-79028bcc1e8b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:14.840765 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1f1ea-d5e6-4c15-bfd3-79028bcc1e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:14.860078 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-2cc1f1ea-d5e6-4c15-bfd3-79028bcc1e8b None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:14.861695 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2cc1f1ea-d5e6-4c15-bfd3-79028bcc1e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:14.925116 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 922/1831] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:14.935500 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b0ffc-9bb7-45b7-84f8-288c3e6bd76d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:14.937019 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8f5b0ffc-9bb7-45b7-84f8-288c3e6bd76d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:14.979095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1f1ea-d5e6-4c15-bfd3-79028bcc1e8b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:14.980730 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2cc1f1ea-d5e6-4c15-bfd3-79028bcc1e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:14.996817 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1f1ea-d5e6-4c15-bfd3-79028bcc1e8b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:15.002386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0262a0f-2d12-4878-b92c-ea99605da52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:15.003421 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0262a0f-2d12-4878-b92c-ea99605da52a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:15.003706 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0262a0f-2d12-4878-b92c-ea99605da52a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:15.003982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0262a0f-2d12-4878-b92c-ea99605da52a admin admin] PATH_INFO: `/v3/projects/7ffd5a842fa449a5b1a9375220bfede8` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:15.005308 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0262a0f-2d12-4878-b92c-ea99605da52a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ffd5a842fa449a5b1a9375220bfede8)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:15.007242 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0262a0f-2d12-4878-b92c-ea99605da52a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:15.007359 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b0ffc-9bb7-45b7-84f8-288c3e6bd76d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:15.011556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff9463-a86a-4412-982f-4878a2e859de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:15.012291 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff9463-a86a-4412-982f-4878a2e859de service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:15.012711 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff9463-a86a-4412-982f-4878a2e859de service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:15.013049 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff9463-a86a-4412-982f-4878a2e859de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:15.022890 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bff9463-a86a-4412-982f-4878a2e859de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:15.024516 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bff9463-a86a-4412-982f-4878a2e859de service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:15.026387 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:15.034552 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 910/1832] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:42:15.121008 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a0262a0f-2d12-4878-b92c-ea99605da52a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ffd5a842fa449a5b1a9375220bfede8', 'request_id': 'req-a0262a0f-2d12-4878-b92c-ea99605da52a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:15.126674 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a0262a0f-2d12-4878-b92c-ea99605da52a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ffd5a842fa449a5b1a9375220bfede8', 'request_id': 'req-a0262a0f-2d12-4878-b92c-ea99605da52a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:15.190292 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a0262a0f-2d12-4878-b92c-ea99605da52a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ffd5a842fa449a5b1a9375220bfede8', 'request_id': 'req-a0262a0f-2d12-4878-b92c-ea99605da52a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:15.203199 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a0262a0f-2d12-4878-b92c-ea99605da52a admin admin] The token cache is being invalidate because project 7ffd5a842fa449a5b1a9375220bfede8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:42:15.203583 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a0262a0f-2d12-4878-b92c-ea99605da52a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:15.204256 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a0262a0f-2d12-4878-b92c-ea99605da52a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:15.205850 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 923/1833] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:42:14 2022] DELETE /identity/v3/projects/7ffd5a842fa449a5b1a9375220bfede8 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:42:15.209784 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff9463-a86a-4412-982f-4878a2e859de service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:15.220893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff9463-a86a-4412-982f-4878a2e859de service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:15.328088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945f8ce-e463-41cc-bf29-033c4bc58ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:15.328832 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945f8ce-e463-41cc-bf29-033c4bc58ba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:15.329473 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945f8ce-e463-41cc-bf29-033c4bc58ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:15.329831 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945f8ce-e463-41cc-bf29-033c4bc58ba3 admin admin] PATH_INFO: `/v3/users/57b031573a134d5ab5efd6290ea9a110` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:15.331605 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945f8ce-e463-41cc-bf29-033c4bc58ba3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57b031573a134d5ab5efd6290ea9a110)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:15.334004 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945f8ce-e463-41cc-bf29-033c4bc58ba3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:15.556063 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0262a0f-2d12-4878-b92c-ea99605da52a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:15.558679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a0262a0f-2d12-4878-b92c-ea99605da52a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:15.562036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e945f8ce-e463-41cc-bf29-033c4bc58ba3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57b031573a134d5ab5efd6290ea9a110'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:15.581090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0262a0f-2d12-4878-b92c-ea99605da52a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:15.586523 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd940167-ff30-4923-96c5-3a71a9c9418c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:15.587215 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd940167-ff30-4923-96c5-3a71a9c9418c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:15.587596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd940167-ff30-4923-96c5-3a71a9c9418c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:15.587970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd940167-ff30-4923-96c5-3a71a9c9418c admin admin] PATH_INFO: `/v3/projects/79b917e5c2ff48f3968b6c55e54a69b2` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:15.589573 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd940167-ff30-4923-96c5-3a71a9c9418c admin admin] RBAC: Authorizing `identity:delete_project(project_id=79b917e5c2ff48f3968b6c55e54a69b2)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:15.592282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd940167-ff30-4923-96c5-3a71a9c9418c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:15.705177 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e945f8ce-e463-41cc-bf29-033c4bc58ba3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57b031573a134d5ab5efd6290ea9a110'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:15.705841 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e945f8ce-e463-41cc-bf29-033c4bc58ba3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57b031573a134d5ab5efd6290ea9a110'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:15.706467 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e945f8ce-e463-41cc-bf29-033c4bc58ba3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57b031573a134d5ab5efd6290ea9a110'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:15.714625 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e945f8ce-e463-41cc-bf29-033c4bc58ba3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57b031573a134d5ab5efd6290ea9a110'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:15.733213 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 911/1834] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:42:15 2022] DELETE /identity/v3/users/57b031573a134d5ab5efd6290ea9a110 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:15.733756 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bd940167-ff30-4923-96c5-3a71a9c9418c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79b917e5c2ff48f3968b6c55e54a69b2', 'request_id': 'req-bd940167-ff30-4923-96c5-3a71a9c9418c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:15.741016 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bd940167-ff30-4923-96c5-3a71a9c9418c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79b917e5c2ff48f3968b6c55e54a69b2', 'request_id': 'req-bd940167-ff30-4923-96c5-3a71a9c9418c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:15.796308 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bd940167-ff30-4923-96c5-3a71a9c9418c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79b917e5c2ff48f3968b6c55e54a69b2', 'request_id': 'req-bd940167-ff30-4923-96c5-3a71a9c9418c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:15.810855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bd940167-ff30-4923-96c5-3a71a9c9418c admin admin] The token cache is being invalidate because project 79b917e5c2ff48f3968b6c55e54a69b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:42:15.811188 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bd940167-ff30-4923-96c5-3a71a9c9418c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:15.812034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bd940167-ff30-4923-96c5-3a71a9c9418c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:15.813695 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 924/1835] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:42:15 2022] DELETE /identity/v3/projects/79b917e5c2ff48f3968b6c55e54a69b2 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:15.817725 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945f8ce-e463-41cc-bf29-033c4bc58ba3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:15.819244 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e945f8ce-e463-41cc-bf29-033c4bc58ba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:15.883697 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945f8ce-e463-41cc-bf29-033c4bc58ba3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:15.889310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662f805-1e29-47a8-af4d-41f7e8f1c226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:15.889952 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662f805-1e29-47a8-af4d-41f7e8f1c226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:15.890231 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662f805-1e29-47a8-af4d-41f7e8f1c226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:15.890502 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662f805-1e29-47a8-af4d-41f7e8f1c226 admin admin] PATH_INFO: `/v3/users/458998dee69e47848785710e4b03550b` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:15.891653 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8662f805-1e29-47a8-af4d-41f7e8f1c226 admin admin] RBAC: Authorizing `identity:delete_user(user_id=458998dee69e47848785710e4b03550b)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:15.893614 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8662f805-1e29-47a8-af4d-41f7e8f1c226 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:16.092312 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8662f805-1e29-47a8-af4d-41f7e8f1c226 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '458998dee69e47848785710e4b03550b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:16.167958 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8662f805-1e29-47a8-af4d-41f7e8f1c226 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '458998dee69e47848785710e4b03550b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:16.168614 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8662f805-1e29-47a8-af4d-41f7e8f1c226 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '458998dee69e47848785710e4b03550b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:16.169266 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8662f805-1e29-47a8-af4d-41f7e8f1c226 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '458998dee69e47848785710e4b03550b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:16.180090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8662f805-1e29-47a8-af4d-41f7e8f1c226 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '458998dee69e47848785710e4b03550b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:16.220605 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 912/1836] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:42:15 2022] DELETE /identity/v3/users/458998dee69e47848785710e4b03550b => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:16.451883 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd940167-ff30-4923-96c5-3a71a9c9418c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:16.453872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bd940167-ff30-4923-96c5-3a71a9c9418c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:16.563549 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd940167-ff30-4923-96c5-3a71a9c9418c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:16.650312 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4950e51-227f-45e0-aa72-04b33f489087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:16.650973 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4950e51-227f-45e0-aa72-04b33f489087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:16.651348 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4950e51-227f-45e0-aa72-04b33f489087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:16.651629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4950e51-227f-45e0-aa72-04b33f489087 admin admin] PATH_INFO: `/v3/projects/461df23abd9b454b94e4f6f5bb542b39` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:16.653121 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4950e51-227f-45e0-aa72-04b33f489087 admin admin] RBAC: Authorizing `identity:delete_project(project_id=461df23abd9b454b94e4f6f5bb542b39)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:16.655266 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4950e51-227f-45e0-aa72-04b33f489087 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:16.677499 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:16.831029 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a4950e51-227f-45e0-aa72-04b33f489087 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '461df23abd9b454b94e4f6f5bb542b39', 'request_id': 'req-a4950e51-227f-45e0-aa72-04b33f489087'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:16.838113 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a4950e51-227f-45e0-aa72-04b33f489087 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '461df23abd9b454b94e4f6f5bb542b39', 'request_id': 'req-a4950e51-227f-45e0-aa72-04b33f489087'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:16.933732 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a4950e51-227f-45e0-aa72-04b33f489087 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '461df23abd9b454b94e4f6f5bb542b39', 'request_id': 'req-a4950e51-227f-45e0-aa72-04b33f489087'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:16.942711 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662f805-1e29-47a8-af4d-41f7e8f1c226 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:16.944183 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8662f805-1e29-47a8-af4d-41f7e8f1c226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:16.960019 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a4950e51-227f-45e0-aa72-04b33f489087 admin admin] The token cache is being invalidate because project 461df23abd9b454b94e4f6f5bb542b39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:42:16.960661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a4950e51-227f-45e0-aa72-04b33f489087 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:16.961787 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a4950e51-227f-45e0-aa72-04b33f489087 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:16.972327 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 925/1837] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:42:16 2022] DELETE /identity/v3/projects/461df23abd9b454b94e4f6f5bb542b39 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:16.977847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e3acf-9e23-46a6-97cc-eeca7123869e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:16.978362 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e3acf-9e23-46a6-97cc-eeca7123869e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:16.978867 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e3acf-9e23-46a6-97cc-eeca7123869e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:16.985824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662f805-1e29-47a8-af4d-41f7e8f1c226 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:17.033467 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-464e3acf-9e23-46a6-97cc-eeca7123869e None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:17.041188 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-464e3acf-9e23-46a6-97cc-eeca7123869e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:17.234237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68368c7b-467f-4b4e-a891-ee4b337fcc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:17.234904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68368c7b-467f-4b4e-a891-ee4b337fcc0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:17.235203 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68368c7b-467f-4b4e-a891-ee4b337fcc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:17.238178 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68368c7b-467f-4b4e-a891-ee4b337fcc0c admin admin] PATH_INFO: `/v3/projects/eb0d794df07d424091a0c59a86c4adca` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:17.240255 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68368c7b-467f-4b4e-a891-ee4b337fcc0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb0d794df07d424091a0c59a86c4adca)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:17.242667 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68368c7b-467f-4b4e-a891-ee4b337fcc0c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:17.251255 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 926/1838] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:17.252835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c5da8-b353-4ce9-9212-06ec3a1d96c8 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:17.253129 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c5da8-b353-4ce9-9212-06ec3a1d96c8 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:17.256280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c5da8-b353-4ce9-9212-06ec3a1d96c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:17.274870 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0c7c5da8-b353-4ce9-9212-06ec3a1d96c8 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:17.276440 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0c7c5da8-b353-4ce9-9212-06ec3a1d96c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:17.291458 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 927/1839] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:17.292247 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7c5da8-b353-4ce9-9212-06ec3a1d96c8 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:17.308232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7c5da8-b353-4ce9-9212-06ec3a1d96c8 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:17.316602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ab25f-1fc5-4bcb-9655-545aa95d0d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:17.319050 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ab25f-1fc5-4bcb-9655-545aa95d0d61 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:17.319852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ab25f-1fc5-4bcb-9655-545aa95d0d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:17.321252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ab25f-1fc5-4bcb-9655-545aa95d0d61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:17.323691 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ab25f-1fc5-4bcb-9655-545aa95d0d61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:42:17.324087 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ab25f-1fc5-4bcb-9655-545aa95d0d61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:17.326157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ab25f-1fc5-4bcb-9655-545aa95d0d61 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:17.343579 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 928/1840] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:42:17.346861 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ab25f-1fc5-4bcb-9655-545aa95d0d61 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:17.358758 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ab25f-1fc5-4bcb-9655-545aa95d0d61 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:17.364694 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817a0f8-1d05-4aa6-820c-4948d43c659d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:17.365497 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817a0f8-1d05-4aa6-820c-4948d43c659d admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:17.365777 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817a0f8-1d05-4aa6-820c-4948d43c659d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:17.366051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817a0f8-1d05-4aa6-820c-4948d43c659d admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:17.366676 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7817a0f8-1d05-4aa6-820c-4948d43c659d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:17.370496 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7817a0f8-1d05-4aa6-820c-4948d43c659d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:17.384630 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-68368c7b-467f-4b4e-a891-ee4b337fcc0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb0d794df07d424091a0c59a86c4adca', 'request_id': 'req-68368c7b-467f-4b4e-a891-ee4b337fcc0c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:17.390380 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-68368c7b-467f-4b4e-a891-ee4b337fcc0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb0d794df07d424091a0c59a86c4adca', 'request_id': 'req-68368c7b-467f-4b4e-a891-ee4b337fcc0c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:17.495127 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-68368c7b-467f-4b4e-a891-ee4b337fcc0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb0d794df07d424091a0c59a86c4adca', 'request_id': 'req-68368c7b-467f-4b4e-a891-ee4b337fcc0c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:17.507811 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 929/1841] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:42:17 2022] POST /identity/v3/projects => generated 377 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:42:17.509346 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-68368c7b-467f-4b4e-a891-ee4b337fcc0c admin admin] The token cache is being invalidate because project eb0d794df07d424091a0c59a86c4adca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:42:17.509642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-68368c7b-467f-4b4e-a891-ee4b337fcc0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:17.509888 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817a0f8-1d05-4aa6-820c-4948d43c659d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:17.510314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-68368c7b-467f-4b4e-a891-ee4b337fcc0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:17.512235 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 913/1842] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:42:16 2022] DELETE /identity/v3/projects/eb0d794df07d424091a0c59a86c4adca => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:17.517739 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2526fd-13f0-4c2c-954e-5f3c1921a735 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:17.518036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2526fd-13f0-4c2c-954e-5f3c1921a735 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:17.518379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2526fd-13f0-4c2c-954e-5f3c1921a735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:17.540456 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8e2526fd-13f0-4c2c-954e-5f3c1921a735 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:17.541685 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8e2526fd-13f0-4c2c-954e-5f3c1921a735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:17.577967 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817a0f8-1d05-4aa6-820c-4948d43c659d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:17.582908 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd45442-97d2-4515-a07f-8af830befb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:17.583682 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd45442-97d2-4515-a07f-8af830befb1f admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:17.583970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd45442-97d2-4515-a07f-8af830befb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:17.584243 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd45442-97d2-4515-a07f-8af830befb1f admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:17.584863 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd45442-97d2-4515-a07f-8af830befb1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:17.586808 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd45442-97d2-4515-a07f-8af830befb1f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:17.606779 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 914/1843] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:42:17.616403 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e706015-e608-436b-9fcc-15f5d91883f1 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:17.616790 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e706015-e608-436b-9fcc-15f5d91883f1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:17.617067 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e706015-e608-436b-9fcc-15f5d91883f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:17.637493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4e706015-e608-436b-9fcc-15f5d91883f1 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:17.638703 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4e706015-e608-436b-9fcc-15f5d91883f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:17.647100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 915/1844] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:17.651149 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e706015-e608-436b-9fcc-15f5d91883f1 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:17.660075 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e706015-e608-436b-9fcc-15f5d91883f1 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:17.665331 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc30c4e-7df5-47cc-ae72-f098f66105cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:17.665913 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc30c4e-7df5-47cc-ae72-f098f66105cf admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:17.666192 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc30c4e-7df5-47cc-ae72-f098f66105cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:17.666465 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc30c4e-7df5-47cc-ae72-f098f66105cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:17.667104 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc30c4e-7df5-47cc-ae72-f098f66105cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:42:17.667452 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc30c4e-7df5-47cc-ae72-f098f66105cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:17.669045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc30c4e-7df5-47cc-ae72-f098f66105cf admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:17.680534 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 916/1845] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:42:17.685817 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 930/1846] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:42:17 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:42:17.685950 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc30c4e-7df5-47cc-ae72-f098f66105cf admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:17.686786 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd45442-97d2-4515-a07f-8af830befb1f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:17.692130 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9dd45442-97d2-4515-a07f-8af830befb1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:17.695644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc30c4e-7df5-47cc-ae72-f098f66105cf admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:17.701190 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd45442-97d2-4515-a07f-8af830befb1f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:17.701463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a277a1a-b2b3-4d94-96cd-494562aba7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:17.702065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a277a1a-b2b3-4d94-96cd-494562aba7f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:17.702351 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a277a1a-b2b3-4d94-96cd-494562aba7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:17.702628 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a277a1a-b2b3-4d94-96cd-494562aba7f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:17.703292 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a277a1a-b2b3-4d94-96cd-494562aba7f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:17.705177 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a277a1a-b2b3-4d94-96cd-494562aba7f9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:17.705904 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792dd04-3469-4b88-8a2c-a95cc7ebb57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:17.706505 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792dd04-3469-4b88-8a2c-a95cc7ebb57f admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:17.706788 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792dd04-3469-4b88-8a2c-a95cc7ebb57f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:17.707069 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792dd04-3469-4b88-8a2c-a95cc7ebb57f admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:17.708298 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792dd04-3469-4b88-8a2c-a95cc7ebb57f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:17.710362 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792dd04-3469-4b88-8a2c-a95cc7ebb57f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:17.721033 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 931/1847] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:42:17.725485 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792dd04-3469-4b88-8a2c-a95cc7ebb57f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:17.735756 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792dd04-3469-4b88-8a2c-a95cc7ebb57f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:17.740754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ea8a0-794e-4560-a65a-7b787392992a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:17.741388 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ea8a0-794e-4560-a65a-7b787392992a admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:17.741733 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ea8a0-794e-4560-a65a-7b787392992a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:17.741986 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ea8a0-794e-4560-a65a-7b787392992a admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:17.742622 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ea8a0-794e-4560-a65a-7b787392992a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:17.744325 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ea8a0-794e-4560-a65a-7b787392992a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:17.755436 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 932/1848] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:42:17.760342 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ea8a0-794e-4560-a65a-7b787392992a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:17.771311 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ea8a0-794e-4560-a65a-7b787392992a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:17.777633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d40f47-899f-457d-92b5-89e8fb000bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:17.778450 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d40f47-899f-457d-92b5-89e8fb000bf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:17.778736 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d40f47-899f-457d-92b5-89e8fb000bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:17.779008 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d40f47-899f-457d-92b5-89e8fb000bf9 admin admin] PATH_INFO: `/v3/projects/84b90313fb214c428bd651d7a68eae2a/users/7f8d2e7209c44718a59383d4c2bcd855/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:17.799129 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d40f47-899f-457d-92b5-89e8fb000bf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84b90313fb214c428bd651d7a68eae2a, user_id=7f8d2e7209c44718a59383d4c2bcd855, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:17.805328 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d40f47-899f-457d-92b5-89e8fb000bf9 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:17.809337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 917/1849] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:42:17 2022] POST /identity/v3/projects => generated 381 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:42:17.813261 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a277a1a-b2b3-4d94-96cd-494562aba7f9 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:17.886863 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 933/1850] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:42:17 2022] PUT /identity/v3/projects/84b90313fb214c428bd651d7a68eae2a/users/7f8d2e7209c44718a59383d4c2bcd855/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:42:17.899169 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63665eb2-99bb-43fa-88ce-8254c76503c1 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:17.899512 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63665eb2-99bb-43fa-88ce-8254c76503c1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:17.899798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63665eb2-99bb-43fa-88ce-8254c76503c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:17.919649 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a277a1a-b2b3-4d94-96cd-494562aba7f9 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:17.944363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-63665eb2-99bb-43fa-88ce-8254c76503c1 None None] MFA Rules not processed for user `7f8d2e7209c44718a59383d4c2bcd855`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:17.948317 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-63665eb2-99bb-43fa-88ce-8254c76503c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:18.010599 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc760e9-904e-47bd-af11-2b255bbe95d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:18.011389 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc760e9-904e-47bd-af11-2b255bbe95d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:18.011877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc760e9-904e-47bd-af11-2b255bbe95d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:18.012322 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc760e9-904e-47bd-af11-2b255bbe95d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:18.013103 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc760e9-904e-47bd-af11-2b255bbe95d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:18.015220 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc760e9-904e-47bd-af11-2b255bbe95d5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:18.035008 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 934/1851] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:42:18.047200 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63665eb2-99bb-43fa-88ce-8254c76503c1 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:18.048659 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-63665eb2-99bb-43fa-88ce-8254c76503c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:18.111264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 918/1852] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:42:17 2022] POST /identity/v3/users => generated 291 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:42:18.115940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc760e9-904e-47bd-af11-2b255bbe95d5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:18.129720 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc760e9-904e-47bd-af11-2b255bbe95d5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:18.136939 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6674abb-3fdb-47cc-888f-eb806e8efaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:18.137949 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6674abb-3fdb-47cc-888f-eb806e8efaef admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:18.138744 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6674abb-3fdb-47cc-888f-eb806e8efaef admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:18.139321 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6674abb-3fdb-47cc-888f-eb806e8efaef admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:18.140735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6674abb-3fdb-47cc-888f-eb806e8efaef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:18.142671 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63665eb2-99bb-43fa-88ce-8254c76503c1 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:18.143298 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6674abb-3fdb-47cc-888f-eb806e8efaef admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:18.147971 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5173617-56b3-4845-b5e4-0fa66b07abac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35dd27fd356241b885706375d880222f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:18.148921 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5173617-56b3-4845-b5e4-0fa66b07abac service glance] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:18.149088 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5173617-56b3-4845-b5e4-0fa66b07abac service glance] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:18.149530 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5173617-56b3-4845-b5e4-0fa66b07abac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:18.159562 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5173617-56b3-4845-b5e4-0fa66b07abac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:18.162138 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5173617-56b3-4845-b5e4-0fa66b07abac service glance] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:18.165999 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 919/1853] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:18.167212 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6674abb-3fdb-47cc-888f-eb806e8efaef admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:18.173607 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 935/1854] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:42:18.188607 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6674abb-3fdb-47cc-888f-eb806e8efaef admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:18.198539 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184168b-83e3-4e0a-8e88-b1e7bdc6b075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:18.199160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184168b-83e3-4e0a-8e88-b1e7bdc6b075 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:18.199476 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184168b-83e3-4e0a-8e88-b1e7bdc6b075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:18.199743 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184168b-83e3-4e0a-8e88-b1e7bdc6b075 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:18.200402 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1184168b-83e3-4e0a-8e88-b1e7bdc6b075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:18.202061 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1184168b-83e3-4e0a-8e88-b1e7bdc6b075 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:18.213955 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 920/1855] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:18.218549 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5173617-56b3-4845-b5e4-0fa66b07abac service glance] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:18.238053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5173617-56b3-4845-b5e4-0fa66b07abac service glance] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:18.245770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09c396-6797-42f4-b24c-9b658eb486a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:18.246448 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09c396-6797-42f4-b24c-9b658eb486a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:18.246727 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09c396-6797-42f4-b24c-9b658eb486a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:18.247001 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09c396-6797-42f4-b24c-9b658eb486a2 admin admin] PATH_INFO: `/v3/projects/a45cd1c710b9489db604ca3ee418c7bd/users/e77e8d0d4980439fa00cd1efaf1a9dd2/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:18.272773 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09c396-6797-42f4-b24c-9b658eb486a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a45cd1c710b9489db604ca3ee418c7bd, user_id=e77e8d0d4980439fa00cd1efaf1a9dd2, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:18.282337 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09c396-6797-42f4-b24c-9b658eb486a2 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:18.368375 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 936/1856] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:42:18 2022] PUT /identity/v3/projects/a45cd1c710b9489db604ca3ee418c7bd/users/e77e8d0d4980439fa00cd1efaf1a9dd2/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:42:18.378584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e3559-80d6-4e9b-bee7-f24410a585cf None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:18.379279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e3559-80d6-4e9b-bee7-f24410a585cf None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:18.380331 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e3559-80d6-4e9b-bee7-f24410a585cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:18.422627 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0e0e3559-80d6-4e9b-bee7-f24410a585cf None None] MFA Rules not processed for user `e77e8d0d4980439fa00cd1efaf1a9dd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:18.423941 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0e0e3559-80d6-4e9b-bee7-f24410a585cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:18.446879 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:18.499336 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 921/1857] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:18.503983 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09c396-6797-42f4-b24c-9b658eb486a2 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:18.570841 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09c396-6797-42f4-b24c-9b658eb486a2 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:18.575741 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51df3af-6bed-45c7-90ce-f2a04db92cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:18.576348 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51df3af-6bed-45c7-90ce-f2a04db92cbc admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:18.576626 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51df3af-6bed-45c7-90ce-f2a04db92cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:18.576895 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51df3af-6bed-45c7-90ce-f2a04db92cbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:18.577546 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51df3af-6bed-45c7-90ce-f2a04db92cbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:18.579496 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51df3af-6bed-45c7-90ce-f2a04db92cbc admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:18.619754 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:18.763650 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 937/1858] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:42:18 2022] POST /identity/v3/projects => generated 385 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:42:18.768264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e3559-80d6-4e9b-bee7-f24410a585cf None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:18.778729 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e3559-80d6-4e9b-bee7-f24410a585cf None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:18.833762 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1bd122-2894-4832-840f-90f658566b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:18.834296 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bd122-2894-4832-840f-90f658566b66 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:18.834580 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bd122-2894-4832-840f-90f658566b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:18.834900 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bd122-2894-4832-840f-90f658566b66 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:18.835522 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1bd122-2894-4832-840f-90f658566b66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:18.837741 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1bd122-2894-4832-840f-90f658566b66 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:18.925886 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 922/1859] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:42:18 2022] POST /identity/v3/users => generated 299 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:42:18.929250 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51df3af-6bed-45c7-90ce-f2a04db92cbc admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:18.944988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51df3af-6bed-45c7-90ce-f2a04db92cbc admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:18.950632 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05a4b2-5148-47f7-a9b0-ff8d7384f4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:18.951444 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05a4b2-5148-47f7-a9b0-ff8d7384f4eb admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:18.951718 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05a4b2-5148-47f7-a9b0-ff8d7384f4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:18.952621 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05a4b2-5148-47f7-a9b0-ff8d7384f4eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:18.952828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05a4b2-5148-47f7-a9b0-ff8d7384f4eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:18.954685 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05a4b2-5148-47f7-a9b0-ff8d7384f4eb admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:18.967337 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 938/1860] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:42:18.971975 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1bd122-2894-4832-840f-90f658566b66 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:18.983073 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1bd122-2894-4832-840f-90f658566b66 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:18.988006 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986d546-66fb-4586-893f-6a9f6229db4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:18.988664 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986d546-66fb-4586-893f-6a9f6229db4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:18.989887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986d546-66fb-4586-893f-6a9f6229db4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:18.990180 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986d546-66fb-4586-893f-6a9f6229db4b admin admin] PATH_INFO: `/v3/projects/b676ecec8da647eb9ce27be794f2a990/users/93717f012b1e4af39aeb0486cd8e4dfc/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:19.005791 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986d546-66fb-4586-893f-6a9f6229db4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b676ecec8da647eb9ce27be794f2a990, user_id=93717f012b1e4af39aeb0486cd8e4dfc, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:19.008481 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986d546-66fb-4586-893f-6a9f6229db4b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:19.059628 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 923/1861] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:42:18 2022] PUT /identity/v3/projects/b676ecec8da647eb9ce27be794f2a990/users/93717f012b1e4af39aeb0486cd8e4dfc/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:19.063594 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05a4b2-5148-47f7-a9b0-ff8d7384f4eb admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:19.074237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05a4b2-5148-47f7-a9b0-ff8d7384f4eb admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:19.157499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e930e-994f-47c2-ad6a-87ce46c48543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:19.158241 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e930e-994f-47c2-ad6a-87ce46c48543 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:19.158596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e930e-994f-47c2-ad6a-87ce46c48543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:19.158956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e930e-994f-47c2-ad6a-87ce46c48543 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:19.159689 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e930e-994f-47c2-ad6a-87ce46c48543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:19.168107 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e930e-994f-47c2-ad6a-87ce46c48543 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:19.172312 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 939/1862] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:19.176593 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986d546-66fb-4586-893f-6a9f6229db4b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:19.187630 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986d546-66fb-4586-893f-6a9f6229db4b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:19.194281 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203e41d-455a-4af2-8fdd-b61c466427dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:19.194281 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203e41d-455a-4af2-8fdd-b61c466427dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:19.194281 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203e41d-455a-4af2-8fdd-b61c466427dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:19.194281 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203e41d-455a-4af2-8fdd-b61c466427dd admin admin] PATH_INFO: `/v3/projects/b676ecec8da647eb9ce27be794f2a990/users/93717f012b1e4af39aeb0486cd8e4dfc/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:19.196032 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4203e41d-455a-4af2-8fdd-b61c466427dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b676ecec8da647eb9ce27be794f2a990, user_id=93717f012b1e4af39aeb0486cd8e4dfc, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:19.198644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4203e41d-455a-4af2-8fdd-b61c466427dd admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:19.211263 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 924/1863] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:42:19 2022] PUT /identity/v3/projects/b676ecec8da647eb9ce27be794f2a990/users/93717f012b1e4af39aeb0486cd8e4dfc/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:19.221313 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed52a82-fd83-49af-9227-f70ddc49acf5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:19.221716 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed52a82-fd83-49af-9227-f70ddc49acf5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:19.222130 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed52a82-fd83-49af-9227-f70ddc49acf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:19.257465 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0ed52a82-fd83-49af-9227-f70ddc49acf5 None None] MFA Rules not processed for user `93717f012b1e4af39aeb0486cd8e4dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:19.258823 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0ed52a82-fd83-49af-9227-f70ddc49acf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:19.280958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:19.335907 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 940/1864] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:19.356706 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203e41d-455a-4af2-8fdd-b61c466427dd admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:19.358520 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4203e41d-455a-4af2-8fdd-b61c466427dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:19.417353 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203e41d-455a-4af2-8fdd-b61c466427dd admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:19.421152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bccc613-2f3f-418b-92ba-156ee0749d35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:19.421715 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bccc613-2f3f-418b-92ba-156ee0749d35 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:19.421998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bccc613-2f3f-418b-92ba-156ee0749d35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:19.422268 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bccc613-2f3f-418b-92ba-156ee0749d35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:19.481087 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bccc613-2f3f-418b-92ba-156ee0749d35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:19.482735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bccc613-2f3f-418b-92ba-156ee0749d35 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:19.494606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 925/1865] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:42:19.887879 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:20.664415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:20.734367 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:21.339227 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:21.368152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:21.898683 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:21.929534 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:22.077223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:22.105346 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:22.191260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:22.230900 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:27.944778 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed52a82-fd83-49af-9227-f70ddc49acf5 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:27.946272 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0ed52a82-fd83-49af-9227-f70ddc49acf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:27.960307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed52a82-fd83-49af-9227-f70ddc49acf5 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:27.964428 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25243f4-c0c0-4b35-8564-dcf93bec33aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:27.965156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25243f4-c0c0-4b35-8564-dcf93bec33aa service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:27.965500 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25243f4-c0c0-4b35-8564-dcf93bec33aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:27.965777 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25243f4-c0c0-4b35-8564-dcf93bec33aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:27.981349 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25243f4-c0c0-4b35-8564-dcf93bec33aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:27.983361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25243f4-c0c0-4b35-8564-dcf93bec33aa service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:27.996216 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 941/1866] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:27 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:42:32.344894 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=42 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:34.129219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bccc613-2f3f-418b-92ba-156ee0749d35 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:34.192833 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bccc613-2f3f-418b-92ba-156ee0749d35 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:34.198458 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936edbdf-b722-4e1d-9992-0e2d3fc0676b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:34.199973 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936edbdf-b722-4e1d-9992-0e2d3fc0676b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:34.200444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936edbdf-b722-4e1d-9992-0e2d3fc0676b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:34.200922 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936edbdf-b722-4e1d-9992-0e2d3fc0676b admin admin] PATH_INFO: `/v3/users/e77e8d0d4980439fa00cd1efaf1a9dd2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:34.202198 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936edbdf-b722-4e1d-9992-0e2d3fc0676b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e77e8d0d4980439fa00cd1efaf1a9dd2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:34.204279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936edbdf-b722-4e1d-9992-0e2d3fc0676b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:34.416787 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-936edbdf-b722-4e1d-9992-0e2d3fc0676b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e77e8d0d4980439fa00cd1efaf1a9dd2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:34.488037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-936edbdf-b722-4e1d-9992-0e2d3fc0676b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e77e8d0d4980439fa00cd1efaf1a9dd2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:34.490346 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-936edbdf-b722-4e1d-9992-0e2d3fc0676b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e77e8d0d4980439fa00cd1efaf1a9dd2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:34.490346 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-936edbdf-b722-4e1d-9992-0e2d3fc0676b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e77e8d0d4980439fa00cd1efaf1a9dd2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:34.497842 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-936edbdf-b722-4e1d-9992-0e2d3fc0676b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e77e8d0d4980439fa00cd1efaf1a9dd2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:34.516321 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 926/1867] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:42:34 2022] DELETE /identity/v3/users/e77e8d0d4980439fa00cd1efaf1a9dd2 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:34.520926 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244636d7-ce62-4e62-b38f-57c38392cf86 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:34.521404 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244636d7-ce62-4e62-b38f-57c38392cf86 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:34.521521 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244636d7-ce62-4e62-b38f-57c38392cf86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:34.541328 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-244636d7-ce62-4e62-b38f-57c38392cf86 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:34.542636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-244636d7-ce62-4e62-b38f-57c38392cf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:34.602784 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 942/1868] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:34.612787 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936edbdf-b722-4e1d-9992-0e2d3fc0676b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:34.615216 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-936edbdf-b722-4e1d-9992-0e2d3fc0676b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:34.674510 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936edbdf-b722-4e1d-9992-0e2d3fc0676b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:34.678033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98029740-3d96-4be4-bfed-7b69ac8eb925 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:34.678777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98029740-3d96-4be4-bfed-7b69ac8eb925 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:34.679060 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98029740-3d96-4be4-bfed-7b69ac8eb925 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:34.679429 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98029740-3d96-4be4-bfed-7b69ac8eb925 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:34.688285 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98029740-3d96-4be4-bfed-7b69ac8eb925 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:34.689700 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98029740-3d96-4be4-bfed-7b69ac8eb925 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:34.700274 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 927/1869] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:42:35.328218 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244636d7-ce62-4e62-b38f-57c38392cf86 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:35.329710 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-244636d7-ce62-4e62-b38f-57c38392cf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:35.342166 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244636d7-ce62-4e62-b38f-57c38392cf86 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:35.347819 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c39bf-bbca-4aa1-b061-98cf7c414b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:35.347965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c39bf-bbca-4aa1-b061-98cf7c414b1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:35.348218 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c39bf-bbca-4aa1-b061-98cf7c414b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:35.348494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c39bf-bbca-4aa1-b061-98cf7c414b1f admin admin] PATH_INFO: `/v3/projects/a45cd1c710b9489db604ca3ee418c7bd` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:35.349740 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c39bf-bbca-4aa1-b061-98cf7c414b1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a45cd1c710b9489db604ca3ee418c7bd)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:35.352470 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c39bf-bbca-4aa1-b061-98cf7c414b1f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:35.487969 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-796c39bf-bbca-4aa1-b061-98cf7c414b1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a45cd1c710b9489db604ca3ee418c7bd', 'request_id': 'req-796c39bf-bbca-4aa1-b061-98cf7c414b1f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:35.495009 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-796c39bf-bbca-4aa1-b061-98cf7c414b1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a45cd1c710b9489db604ca3ee418c7bd', 'request_id': 'req-796c39bf-bbca-4aa1-b061-98cf7c414b1f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:35.595261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-796c39bf-bbca-4aa1-b061-98cf7c414b1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a45cd1c710b9489db604ca3ee418c7bd', 'request_id': 'req-796c39bf-bbca-4aa1-b061-98cf7c414b1f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:35.608035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-796c39bf-bbca-4aa1-b061-98cf7c414b1f admin admin] The token cache is being invalidate because project a45cd1c710b9489db604ca3ee418c7bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:42:35.609287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-796c39bf-bbca-4aa1-b061-98cf7c414b1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:35.609862 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-796c39bf-bbca-4aa1-b061-98cf7c414b1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:35.611877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 943/1870] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:42:35 2022] DELETE /identity/v3/projects/a45cd1c710b9489db604ca3ee418c7bd => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:35.615707 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98029740-3d96-4be4-bfed-7b69ac8eb925 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:35.626443 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98029740-3d96-4be4-bfed-7b69ac8eb925 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:35.650143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:35.689238 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9aa285-c50a-4e25-9dae-1e88d46140a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:35.689869 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9aa285-c50a-4e25-9dae-1e88d46140a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:35.690147 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9aa285-c50a-4e25-9dae-1e88d46140a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:35.690422 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9aa285-c50a-4e25-9dae-1e88d46140a0 admin admin] PATH_INFO: `/v3/users/93717f012b1e4af39aeb0486cd8e4dfc` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:35.691563 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9aa285-c50a-4e25-9dae-1e88d46140a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93717f012b1e4af39aeb0486cd8e4dfc)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:35.693474 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9aa285-c50a-4e25-9dae-1e88d46140a0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:35.895696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fd9aa285-c50a-4e25-9dae-1e88d46140a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93717f012b1e4af39aeb0486cd8e4dfc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:35.978524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fd9aa285-c50a-4e25-9dae-1e88d46140a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93717f012b1e4af39aeb0486cd8e4dfc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:35.979208 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fd9aa285-c50a-4e25-9dae-1e88d46140a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93717f012b1e4af39aeb0486cd8e4dfc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:35.979839 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fd9aa285-c50a-4e25-9dae-1e88d46140a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93717f012b1e4af39aeb0486cd8e4dfc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:35.987920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fd9aa285-c50a-4e25-9dae-1e88d46140a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93717f012b1e4af39aeb0486cd8e4dfc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:36.006543 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 928/1871] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:42:35 2022] DELETE /identity/v3/users/93717f012b1e4af39aeb0486cd8e4dfc => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:36.501249 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c39bf-bbca-4aa1-b061-98cf7c414b1f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:36.502720 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-796c39bf-bbca-4aa1-b061-98cf7c414b1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:36.517111 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c39bf-bbca-4aa1-b061-98cf7c414b1f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:36.582504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc7915-5ee9-4ef7-ab88-d56739eadf08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:36.583155 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc7915-5ee9-4ef7-ab88-d56739eadf08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:36.583467 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc7915-5ee9-4ef7-ab88-d56739eadf08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:36.583746 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc7915-5ee9-4ef7-ab88-d56739eadf08 admin admin] PATH_INFO: `/v3/projects/b676ecec8da647eb9ce27be794f2a990` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:36.584902 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cc7915-5ee9-4ef7-ab88-d56739eadf08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b676ecec8da647eb9ce27be794f2a990)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:36.587563 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cc7915-5ee9-4ef7-ab88-d56739eadf08 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:36.589074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:36.694310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-81cc7915-5ee9-4ef7-ab88-d56739eadf08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b676ecec8da647eb9ce27be794f2a990', 'request_id': 'req-81cc7915-5ee9-4ef7-ab88-d56739eadf08'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:36.700322 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-81cc7915-5ee9-4ef7-ab88-d56739eadf08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b676ecec8da647eb9ce27be794f2a990', 'request_id': 'req-81cc7915-5ee9-4ef7-ab88-d56739eadf08'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:36.801961 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-81cc7915-5ee9-4ef7-ab88-d56739eadf08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b676ecec8da647eb9ce27be794f2a990', 'request_id': 'req-81cc7915-5ee9-4ef7-ab88-d56739eadf08'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:36.813792 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-81cc7915-5ee9-4ef7-ab88-d56739eadf08 admin admin] The token cache is being invalidate because project b676ecec8da647eb9ce27be794f2a990 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:42:36.814080 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-81cc7915-5ee9-4ef7-ab88-d56739eadf08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:36.814728 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-81cc7915-5ee9-4ef7-ab88-d56739eadf08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:36.816250 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 944/1872] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:42:36 2022] DELETE /identity/v3/projects/b676ecec8da647eb9ce27be794f2a990 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:36.822988 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4564e9d-ce9c-4c4e-a0d1-1ce16e3463ff None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:36.823496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4564e9d-ce9c-4c4e-a0d1-1ce16e3463ff None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:36.823608 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4564e9d-ce9c-4c4e-a0d1-1ce16e3463ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:36.843441 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c4564e9d-ce9c-4c4e-a0d1-1ce16e3463ff None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:36.844658 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c4564e9d-ce9c-4c4e-a0d1-1ce16e3463ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:36.904668 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 929/1873] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:36.913153 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191639a-c223-4391-9689-c3f5017ce0eb None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:36.913444 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191639a-c223-4391-9689-c3f5017ce0eb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:36.913802 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191639a-c223-4391-9689-c3f5017ce0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:36.933402 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6191639a-c223-4391-9689-c3f5017ce0eb None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:36.934628 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6191639a-c223-4391-9689-c3f5017ce0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:36.943126 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 945/1874] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:36.948022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4564e9d-ce9c-4c4e-a0d1-1ce16e3463ff None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:36.958445 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4564e9d-ce9c-4c4e-a0d1-1ce16e3463ff None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:36.964059 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d41bd6-c2c9-413e-a002-ba8d70d0a212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:36.964668 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d41bd6-c2c9-413e-a002-ba8d70d0a212 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:36.964946 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d41bd6-c2c9-413e-a002-ba8d70d0a212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:36.965218 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d41bd6-c2c9-413e-a002-ba8d70d0a212 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:36.965857 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d41bd6-c2c9-413e-a002-ba8d70d0a212 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:42:36.966185 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d41bd6-c2c9-413e-a002-ba8d70d0a212 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:36.967843 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d41bd6-c2c9-413e-a002-ba8d70d0a212 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:36.979556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 930/1875] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:42:36.983971 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191639a-c223-4391-9689-c3f5017ce0eb None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:36.994221 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191639a-c223-4391-9689-c3f5017ce0eb None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:36.998848 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86c45a-edbc-48fa-a39c-cb5ea76011c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:36.999481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86c45a-edbc-48fa-a39c-cb5ea76011c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:36.999766 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86c45a-edbc-48fa-a39c-cb5ea76011c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:37.000041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86c45a-edbc-48fa-a39c-cb5ea76011c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:37.000658 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd86c45a-edbc-48fa-a39c-cb5ea76011c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:37.002547 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd86c45a-edbc-48fa-a39c-cb5ea76011c7 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:37.083919 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 946/1876] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:42:36 2022] POST /identity/v3/projects => generated 361 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:42:37.088822 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d41bd6-c2c9-413e-a002-ba8d70d0a212 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:37.098700 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d41bd6-c2c9-413e-a002-ba8d70d0a212 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:37.126673 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:37.157677 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c9f1a-22a9-43c6-bc3f-13c46b06a4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:37.158278 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c9f1a-22a9-43c6-bc3f-13c46b06a4a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:37.158560 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c9f1a-22a9-43c6-bc3f-13c46b06a4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:37.158833 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c9f1a-22a9-43c6-bc3f-13c46b06a4a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:37.159484 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c9f1a-22a9-43c6-bc3f-13c46b06a4a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:37.161400 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c9f1a-22a9-43c6-bc3f-13c46b06a4a8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:37.259774 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 931/1877] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:42:37 2022] POST /identity/v3/users => generated 281 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:42:37.263902 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86c45a-edbc-48fa-a39c-cb5ea76011c7 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:37.274833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86c45a-edbc-48fa-a39c-cb5ea76011c7 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:37.279505 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8206c-6a1b-4f1a-837c-bd6bc68305a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:37.280109 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8206c-6a1b-4f1a-837c-bd6bc68305a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:37.280389 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8206c-6a1b-4f1a-837c-bd6bc68305a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:37.280660 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8206c-6a1b-4f1a-837c-bd6bc68305a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:37.281340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e8206c-6a1b-4f1a-837c-bd6bc68305a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:37.282908 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e8206c-6a1b-4f1a-837c-bd6bc68305a6 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:37.293470 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 947/1878] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:37.297433 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c9f1a-22a9-43c6-bc3f-13c46b06a4a8 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:37.308054 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c9f1a-22a9-43c6-bc3f-13c46b06a4a8 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:37.312669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a53cda1-e76c-4fe1-a69a-22a0084ebcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:37.313267 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53cda1-e76c-4fe1-a69a-22a0084ebcac admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:37.313547 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53cda1-e76c-4fe1-a69a-22a0084ebcac admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:37.313821 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53cda1-e76c-4fe1-a69a-22a0084ebcac admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:37.314456 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a53cda1-e76c-4fe1-a69a-22a0084ebcac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:37.316079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a53cda1-e76c-4fe1-a69a-22a0084ebcac admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:37.326521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 932/1879] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:37.330496 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8206c-6a1b-4f1a-837c-bd6bc68305a6 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:37.340555 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8206c-6a1b-4f1a-837c-bd6bc68305a6 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:37.345354 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2e71df-11b8-4847-9f02-c22e00254b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:37.345997 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e71df-11b8-4847-9f02-c22e00254b57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:37.346277 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e71df-11b8-4847-9f02-c22e00254b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:37.346554 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e71df-11b8-4847-9f02-c22e00254b57 admin admin] PATH_INFO: `/v3/projects/fec1a965a16a4702b29d8949e8f2d1b4/users/93e68daefa0e404789b3167257e0acb8/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:37.361840 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2e71df-11b8-4847-9f02-c22e00254b57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fec1a965a16a4702b29d8949e8f2d1b4, user_id=93e68daefa0e404789b3167257e0acb8, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:37.364526 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2e71df-11b8-4847-9f02-c22e00254b57 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:37.463925 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 948/1880] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:42:37 2022] PUT /identity/v3/projects/fec1a965a16a4702b29d8949e8f2d1b4/users/93e68daefa0e404789b3167257e0acb8/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:37.475503 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246932a0-0e7e-4fe9-ad27-9fff5be2e7c8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:37.475796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246932a0-0e7e-4fe9-ad27-9fff5be2e7c8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:37.476078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246932a0-0e7e-4fe9-ad27-9fff5be2e7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:37.505488 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-246932a0-0e7e-4fe9-ad27-9fff5be2e7c8 None None] MFA Rules not processed for user `93e68daefa0e404789b3167257e0acb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:37.506699 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-246932a0-0e7e-4fe9-ad27-9fff5be2e7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:37.577012 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 933/1881] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:37.612126 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2e71df-11b8-4847-9f02-c22e00254b57 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:37.613917 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6a2e71df-11b8-4847-9f02-c22e00254b57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:37.673718 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2e71df-11b8-4847-9f02-c22e00254b57 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:37.677361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb2bea3-fa0d-40e3-b23d-bfcb4b128c4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:37.677970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2bea3-fa0d-40e3-b23d-bfcb4b128c4d service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:37.678250 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2bea3-fa0d-40e3-b23d-bfcb4b128c4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:37.678532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2bea3-fa0d-40e3-b23d-bfcb4b128c4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:37.686862 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb2bea3-fa0d-40e3-b23d-bfcb4b128c4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:37.688309 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb2bea3-fa0d-40e3-b23d-bfcb4b128c4d service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:37.697499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 949/1882] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:42:37.713039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246932a0-0e7e-4fe9-ad27-9fff5be2e7c8 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:37.774485 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246932a0-0e7e-4fe9-ad27-9fff5be2e7c8 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:37.778987 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb0531-782c-4fad-b519-1afa8e72282c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:37.779640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb0531-782c-4fad-b519-1afa8e72282c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:37.779918 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb0531-782c-4fad-b519-1afa8e72282c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:37.780191 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb0531-782c-4fad-b519-1afa8e72282c admin admin] PATH_INFO: `/v3/users/93e68daefa0e404789b3167257e0acb8` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:37.781307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb0531-782c-4fad-b519-1afa8e72282c admin admin] RBAC: Authorizing `identity:delete_user(user_id=93e68daefa0e404789b3167257e0acb8)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:37.783241 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb0531-782c-4fad-b519-1afa8e72282c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:37.952718 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:37.958299 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-65eb0531-782c-4fad-b519-1afa8e72282c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93e68daefa0e404789b3167257e0acb8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:38.036132 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-65eb0531-782c-4fad-b519-1afa8e72282c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93e68daefa0e404789b3167257e0acb8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:38.036776 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-65eb0531-782c-4fad-b519-1afa8e72282c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93e68daefa0e404789b3167257e0acb8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:38.037385 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-65eb0531-782c-4fad-b519-1afa8e72282c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93e68daefa0e404789b3167257e0acb8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:38.045256 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-65eb0531-782c-4fad-b519-1afa8e72282c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93e68daefa0e404789b3167257e0acb8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:38.062212 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 934/1883] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:42:37 2022] DELETE /identity/v3/users/93e68daefa0e404789b3167257e0acb8 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:38.066707 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b251dc-ef41-44d5-bb0f-8ca7380eec9a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:38.066994 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b251dc-ef41-44d5-bb0f-8ca7380eec9a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:38.067296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b251dc-ef41-44d5-bb0f-8ca7380eec9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:38.086390 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-91b251dc-ef41-44d5-bb0f-8ca7380eec9a None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:38.087718 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-91b251dc-ef41-44d5-bb0f-8ca7380eec9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:38.114005 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:38.150524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 950/1884] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:38.163475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb0531-782c-4fad-b519-1afa8e72282c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:38.165379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-65eb0531-782c-4fad-b519-1afa8e72282c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:38.222243 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb0531-782c-4fad-b519-1afa8e72282c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:38.225665 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a4d8dc-96f9-4611-bce3-b0db0f52b2b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:38.226309 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4d8dc-96f9-4611-bce3-b0db0f52b2b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:38.226904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4d8dc-96f9-4611-bce3-b0db0f52b2b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:38.227193 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4d8dc-96f9-4611-bce3-b0db0f52b2b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:38.236192 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a4d8dc-96f9-4611-bce3-b0db0f52b2b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:38.237610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a4d8dc-96f9-4611-bce3-b0db0f52b2b2 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:38.248130 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 935/1885] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:42:38.706714 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b251dc-ef41-44d5-bb0f-8ca7380eec9a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:38.708629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-91b251dc-ef41-44d5-bb0f-8ca7380eec9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:38.740819 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b251dc-ef41-44d5-bb0f-8ca7380eec9a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:38.743918 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b206b9-4fde-4685-bf20-85964dc1698c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:38.745175 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b206b9-4fde-4685-bf20-85964dc1698c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:38.745455 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b206b9-4fde-4685-bf20-85964dc1698c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:38.746058 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b206b9-4fde-4685-bf20-85964dc1698c admin admin] PATH_INFO: `/v3/projects/fec1a965a16a4702b29d8949e8f2d1b4` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:38.747522 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b206b9-4fde-4685-bf20-85964dc1698c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fec1a965a16a4702b29d8949e8f2d1b4)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:38.750565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b206b9-4fde-4685-bf20-85964dc1698c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:38.905617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-77b206b9-4fde-4685-bf20-85964dc1698c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fec1a965a16a4702b29d8949e8f2d1b4', 'request_id': 'req-77b206b9-4fde-4685-bf20-85964dc1698c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:38.910969 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-77b206b9-4fde-4685-bf20-85964dc1698c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fec1a965a16a4702b29d8949e8f2d1b4', 'request_id': 'req-77b206b9-4fde-4685-bf20-85964dc1698c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:38.978836 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-77b206b9-4fde-4685-bf20-85964dc1698c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fec1a965a16a4702b29d8949e8f2d1b4', 'request_id': 'req-77b206b9-4fde-4685-bf20-85964dc1698c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:38.991339 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-77b206b9-4fde-4685-bf20-85964dc1698c admin admin] The token cache is being invalidate because project fec1a965a16a4702b29d8949e8f2d1b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:42:38.991559 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-77b206b9-4fde-4685-bf20-85964dc1698c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:38.992273 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-77b206b9-4fde-4685-bf20-85964dc1698c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:38.994099 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 951/1886] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:42:38 2022] DELETE /identity/v3/projects/fec1a965a16a4702b29d8949e8f2d1b4 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:38.998749 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d046a-f69d-4fb0-8410-572a190ff666 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:38.999261 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d046a-f69d-4fb0-8410-572a190ff666 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:38.999352 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d046a-f69d-4fb0-8410-572a190ff666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:39.019514 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9a4d046a-f69d-4fb0-8410-572a190ff666 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:39.020722 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9a4d046a-f69d-4fb0-8410-572a190ff666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:39.083076 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 936/1887] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:39.092665 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39fe21-914b-4dcd-9184-31e3093c4f90 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:39.092950 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39fe21-914b-4dcd-9184-31e3093c4f90 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:39.093222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39fe21-914b-4dcd-9184-31e3093c4f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:39.113546 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5e39fe21-914b-4dcd-9184-31e3093c4f90 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:39.114754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5e39fe21-914b-4dcd-9184-31e3093c4f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:39.123501 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 952/1888] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:39.127912 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d046a-f69d-4fb0-8410-572a190ff666 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:39.138887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d046a-f69d-4fb0-8410-572a190ff666 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:39.145955 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b83b77-77e3-44a4-9196-48dfd25c2bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:39.147129 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b83b77-77e3-44a4-9196-48dfd25c2bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:39.147129 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b83b77-77e3-44a4-9196-48dfd25c2bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:39.147129 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b83b77-77e3-44a4-9196-48dfd25c2bb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:39.147904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b83b77-77e3-44a4-9196-48dfd25c2bb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:42:39.148329 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b83b77-77e3-44a4-9196-48dfd25c2bb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:39.149990 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b83b77-77e3-44a4-9196-48dfd25c2bb1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:39.163643 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 937/1889] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:42:39.173170 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39fe21-914b-4dcd-9184-31e3093c4f90 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:39.184452 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39fe21-914b-4dcd-9184-31e3093c4f90 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:39.190127 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6086e-6b2c-4a2f-9e01-f88c7e16c6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:39.190889 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6086e-6b2c-4a2f-9e01-f88c7e16c6e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:39.191273 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6086e-6b2c-4a2f-9e01-f88c7e16c6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:39.191740 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6086e-6b2c-4a2f-9e01-f88c7e16c6e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:39.192444 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c6086e-6b2c-4a2f-9e01-f88c7e16c6e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:39.194765 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c6086e-6b2c-4a2f-9e01-f88c7e16c6e4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:39.353240 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 953/1890] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:42:39 2022] POST /identity/v3/projects => generated 373 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:42:39.357674 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b83b77-77e3-44a4-9196-48dfd25c2bb1 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:39.369799 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b83b77-77e3-44a4-9196-48dfd25c2bb1 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:39.421435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:39.434936 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48a429-8582-4a52-9ef8-409be6285937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:39.435626 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48a429-8582-4a52-9ef8-409be6285937 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:39.435911 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48a429-8582-4a52-9ef8-409be6285937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:39.436209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48a429-8582-4a52-9ef8-409be6285937 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:39.436873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48a429-8582-4a52-9ef8-409be6285937 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:39.438805 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48a429-8582-4a52-9ef8-409be6285937 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:39.545553 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 938/1891] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:42:39 2022] POST /identity/v3/users => generated 287 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:42:39.549901 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6086e-6b2c-4a2f-9e01-f88c7e16c6e4 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:39.560346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6086e-6b2c-4a2f-9e01-f88c7e16c6e4 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:39.565190 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf268c7-608e-47d1-b2d0-137c4ed77154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:39.569434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf268c7-608e-47d1-b2d0-137c4ed77154 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:39.569434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf268c7-608e-47d1-b2d0-137c4ed77154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:39.569434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf268c7-608e-47d1-b2d0-137c4ed77154 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:39.569434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf268c7-608e-47d1-b2d0-137c4ed77154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:39.569434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf268c7-608e-47d1-b2d0-137c4ed77154 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:39.583348 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 954/1892] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:39.584510 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48a429-8582-4a52-9ef8-409be6285937 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:39.595213 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48a429-8582-4a52-9ef8-409be6285937 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:39.599819 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc6f6e-db10-42f7-ab7a-44bd9f237d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:39.600410 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc6f6e-db10-42f7-ab7a-44bd9f237d8d admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:39.600688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc6f6e-db10-42f7-ab7a-44bd9f237d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:39.600956 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc6f6e-db10-42f7-ab7a-44bd9f237d8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:39.602216 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc6f6e-db10-42f7-ab7a-44bd9f237d8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:39.603979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc6f6e-db10-42f7-ab7a-44bd9f237d8d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:39.615332 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 939/1893] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:42:39.619789 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf268c7-608e-47d1-b2d0-137c4ed77154 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:39.631418 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf268c7-608e-47d1-b2d0-137c4ed77154 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:39.635787 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857da56-00ff-48ff-be6a-3ee956d28016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:39.636442 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857da56-00ff-48ff-be6a-3ee956d28016 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:39.636896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857da56-00ff-48ff-be6a-3ee956d28016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:39.637008 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857da56-00ff-48ff-be6a-3ee956d28016 admin admin] PATH_INFO: `/v3/projects/a0f0e852f53e451aa5c64e6f1a8b4125/users/aa8f427dd0a84cb4b94c5797a84d72f2/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:39.653421 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7857da56-00ff-48ff-be6a-3ee956d28016 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0f0e852f53e451aa5c64e6f1a8b4125, user_id=aa8f427dd0a84cb4b94c5797a84d72f2, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:39.656146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7857da56-00ff-48ff-be6a-3ee956d28016 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:39.740021 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 955/1894] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:42:39 2022] PUT /identity/v3/projects/a0f0e852f53e451aa5c64e6f1a8b4125/users/aa8f427dd0a84cb4b94c5797a84d72f2/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:39.751565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1fabf-6754-4b82-88a4-1261f586a052 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:39.751954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1fabf-6754-4b82-88a4-1261f586a052 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:39.752260 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1fabf-6754-4b82-88a4-1261f586a052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:39.781224 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a1f1fabf-6754-4b82-88a4-1261f586a052 None None] MFA Rules not processed for user `aa8f427dd0a84cb4b94c5797a84d72f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:39.782399 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a1f1fabf-6754-4b82-88a4-1261f586a052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:39.848145 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 940/1895] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:39.852978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857da56-00ff-48ff-be6a-3ee956d28016 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:39.904602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:39.917881 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857da56-00ff-48ff-be6a-3ee956d28016 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:39.922586 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebdb97c-3aa5-489c-ab54-6a016f32730e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:39.923229 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdb97c-3aa5-489c-ab54-6a016f32730e admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:39.923515 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdb97c-3aa5-489c-ab54-6a016f32730e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:39.923823 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdb97c-3aa5-489c-ab54-6a016f32730e admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:39.924453 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebdb97c-3aa5-489c-ab54-6a016f32730e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:39.926776 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebdb97c-3aa5-489c-ab54-6a016f32730e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:40.061735 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 956/1896] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:42:39 2022] POST /identity/v3/projects => generated 375 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:42:40.065403 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1fabf-6754-4b82-88a4-1261f586a052 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:40.076872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1fabf-6754-4b82-88a4-1261f586a052 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:40.135519 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148edf20-5b0d-4b22-8842-5e51c808fa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:40.136133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148edf20-5b0d-4b22-8842-5e51c808fa89 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:40.136432 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148edf20-5b0d-4b22-8842-5e51c808fa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:40.136693 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148edf20-5b0d-4b22-8842-5e51c808fa89 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:40.137323 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148edf20-5b0d-4b22-8842-5e51c808fa89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:40.139264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148edf20-5b0d-4b22-8842-5e51c808fa89 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:40.281447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 941/1897] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:42:40 2022] POST /identity/v3/users => generated 294 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:42:40.285524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebdb97c-3aa5-489c-ab54-6a016f32730e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:40.296503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebdb97c-3aa5-489c-ab54-6a016f32730e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:40.301048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c5d286-511b-4599-a0be-ec4f2205d000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:40.301645 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5d286-511b-4599-a0be-ec4f2205d000 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:40.301928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5d286-511b-4599-a0be-ec4f2205d000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:40.302244 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5d286-511b-4599-a0be-ec4f2205d000 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:40.302877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c5d286-511b-4599-a0be-ec4f2205d000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:40.304627 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c5d286-511b-4599-a0be-ec4f2205d000 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:40.314810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 957/1898] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:40.319005 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148edf20-5b0d-4b22-8842-5e51c808fa89 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:40.329202 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148edf20-5b0d-4b22-8842-5e51c808fa89 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:40.334014 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb54dd7-a6be-4917-a909-47fb336ae976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:40.336144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb54dd7-a6be-4917-a909-47fb336ae976 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:40.336144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb54dd7-a6be-4917-a909-47fb336ae976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:40.336144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb54dd7-a6be-4917-a909-47fb336ae976 admin admin] PATH_INFO: `/v3/projects/67fcddbfd9364fdd858ea3d4574a46a5/users/525deea5f74241ff87025f79faf2969c/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:40.352466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb54dd7-a6be-4917-a909-47fb336ae976 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67fcddbfd9364fdd858ea3d4574a46a5, user_id=525deea5f74241ff87025f79faf2969c, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:40.355465 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb54dd7-a6be-4917-a909-47fb336ae976 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:40.453691 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 942/1899] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:42:40 2022] PUT /identity/v3/projects/67fcddbfd9364fdd858ea3d4574a46a5/users/525deea5f74241ff87025f79faf2969c/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:40.457945 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c5d286-511b-4599-a0be-ec4f2205d000 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:40.468613 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c5d286-511b-4599-a0be-ec4f2205d000 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:40.527767 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2dbf0-c446-4732-a048-20f36dc9e4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:40.528576 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2dbf0-c446-4732-a048-20f36dc9e4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:40.528867 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2dbf0-c446-4732-a048-20f36dc9e4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:40.529257 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2dbf0-c446-4732-a048-20f36dc9e4d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:40.529948 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2dbf0-c446-4732-a048-20f36dc9e4d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:40.531569 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2dbf0-c446-4732-a048-20f36dc9e4d6 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:40.543701 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 958/1900] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:42:40.553698 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb54dd7-a6be-4917-a909-47fb336ae976 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:40.566175 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb54dd7-a6be-4917-a909-47fb336ae976 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:40.571258 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d234387-469f-43e6-a391-7ba0544998c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:40.572218 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d234387-469f-43e6-a391-7ba0544998c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:40.572501 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d234387-469f-43e6-a391-7ba0544998c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:40.572802 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d234387-469f-43e6-a391-7ba0544998c6 admin admin] PATH_INFO: `/v3/projects/67fcddbfd9364fdd858ea3d4574a46a5/users/525deea5f74241ff87025f79faf2969c/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:40.574794 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d234387-469f-43e6-a391-7ba0544998c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67fcddbfd9364fdd858ea3d4574a46a5, user_id=525deea5f74241ff87025f79faf2969c, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:40.578160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d234387-469f-43e6-a391-7ba0544998c6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:40.590625 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 943/1901] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:42:40 2022] PUT /identity/v3/projects/67fcddbfd9364fdd858ea3d4574a46a5/users/525deea5f74241ff87025f79faf2969c/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:40.600864 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cae496-4bf2-4e83-b0c3-303a142bf3a9 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:40.601274 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cae496-4bf2-4e83-b0c3-303a142bf3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:40.601637 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cae496-4bf2-4e83-b0c3-303a142bf3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:40.629271 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-88cae496-4bf2-4e83-b0c3-303a142bf3a9 None None] MFA Rules not processed for user `525deea5f74241ff87025f79faf2969c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:40.630447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-88cae496-4bf2-4e83-b0c3-303a142bf3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:40.698079 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 959/1902] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:40.709160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d234387-469f-43e6-a391-7ba0544998c6 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:40.711530 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4d234387-469f-43e6-a391-7ba0544998c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:40.721306 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:40.773330 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d234387-469f-43e6-a391-7ba0544998c6 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:40.777020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d91d4-ba8a-43fc-8bf1-6d16673b3cec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:40.777629 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d91d4-ba8a-43fc-8bf1-6d16673b3cec service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:40.777932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d91d4-ba8a-43fc-8bf1-6d16673b3cec service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:40.778161 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d91d4-ba8a-43fc-8bf1-6d16673b3cec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:40.839655 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d91d4-ba8a-43fc-8bf1-6d16673b3cec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:40.841090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d91d4-ba8a-43fc-8bf1-6d16673b3cec service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:40.850112 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 944/1903] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:42:43.945982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cae496-4bf2-4e83-b0c3-303a142bf3a9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:44.010926 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:44.011084 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d91d4-ba8a-43fc-8bf1-6d16673b3cec service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:44.011084 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e17d91d4-ba8a-43fc-8bf1-6d16673b3cec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:44.040674 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cae496-4bf2-4e83-b0c3-303a142bf3a9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:44.045779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e40d2-3494-48cd-b5bd-8401a6938e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:44.046173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e40d2-3494-48cd-b5bd-8401a6938e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:44.046461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e40d2-3494-48cd-b5bd-8401a6938e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:44.046766 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e40d2-3494-48cd-b5bd-8401a6938e85 admin admin] PATH_INFO: `/v3/users/aa8f427dd0a84cb4b94c5797a84d72f2` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:44.047889 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e40d2-3494-48cd-b5bd-8401a6938e85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa8f427dd0a84cb4b94c5797a84d72f2)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:44.049841 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e40d2-3494-48cd-b5bd-8401a6938e85 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:44.088273 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d91d4-ba8a-43fc-8bf1-6d16673b3cec service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:44.093265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd935a-51b2-49f7-bb7c-a05268a3f8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:44.093945 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd935a-51b2-49f7-bb7c-a05268a3f8ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:44.094226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd935a-51b2-49f7-bb7c-a05268a3f8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:44.094497 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd935a-51b2-49f7-bb7c-a05268a3f8ed admin admin] PATH_INFO: `/v3/users/7f0f57daed0843abbc46a7903ce0b190` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:44.095650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd935a-51b2-49f7-bb7c-a05268a3f8ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f0f57daed0843abbc46a7903ce0b190)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:44.097571 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd935a-51b2-49f7-bb7c-a05268a3f8ed admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:44.246062 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1b8e40d2-3494-48cd-b5bd-8401a6938e85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa8f427dd0a84cb4b94c5797a84d72f2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:44.252822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1b8e40d2-3494-48cd-b5bd-8401a6938e85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa8f427dd0a84cb4b94c5797a84d72f2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:44.330956 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-85dd935a-51b2-49f7-bb7c-a05268a3f8ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f0f57daed0843abbc46a7903ce0b190'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:44.344701 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1b8e40d2-3494-48cd-b5bd-8401a6938e85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa8f427dd0a84cb4b94c5797a84d72f2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:44.345281 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1b8e40d2-3494-48cd-b5bd-8401a6938e85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa8f427dd0a84cb4b94c5797a84d72f2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:44.345893 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1b8e40d2-3494-48cd-b5bd-8401a6938e85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa8f427dd0a84cb4b94c5797a84d72f2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:44.363935 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 960/1904] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:42:43 2022] DELETE /identity/v3/users/aa8f427dd0a84cb4b94c5797a84d72f2 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:44.368489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f2b7d-4876-4b68-8eaf-118ba0ae0444 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:44.368779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f2b7d-4876-4b68-8eaf-118ba0ae0444 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:44.369058 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f2b7d-4876-4b68-8eaf-118ba0ae0444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:44.388477 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-671f2b7d-4876-4b68-8eaf-118ba0ae0444 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:44.389892 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-671f2b7d-4876-4b68-8eaf-118ba0ae0444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:44.413251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-85dd935a-51b2-49f7-bb7c-a05268a3f8ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f0f57daed0843abbc46a7903ce0b190'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:44.413955 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-85dd935a-51b2-49f7-bb7c-a05268a3f8ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f0f57daed0843abbc46a7903ce0b190'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:44.414586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-85dd935a-51b2-49f7-bb7c-a05268a3f8ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f0f57daed0843abbc46a7903ce0b190'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:44.422559 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-85dd935a-51b2-49f7-bb7c-a05268a3f8ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f0f57daed0843abbc46a7903ce0b190'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:44.424030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:44.441644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 945/1905] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:42:44 2022] DELETE /identity/v3/users/7f0f57daed0843abbc46a7903ce0b190 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:42:44.446010 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9649fe-3b18-4f7c-8a15-19ad12871a50 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:44.447131 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9649fe-3b18-4f7c-8a15-19ad12871a50 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:44.447403 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9649fe-3b18-4f7c-8a15-19ad12871a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:44.450402 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 961/1906] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:42:44.460028 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f2b7d-4876-4b68-8eaf-118ba0ae0444 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:44.461417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-671f2b7d-4876-4b68-8eaf-118ba0ae0444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:44.467045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8d9649fe-3b18-4f7c-8a15-19ad12871a50 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:44.468379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8d9649fe-3b18-4f7c-8a15-19ad12871a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:44.526710 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f2b7d-4876-4b68-8eaf-118ba0ae0444 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:44.532433 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 946/1907] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:42:44.543761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9649fe-3b18-4f7c-8a15-19ad12871a50 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:44.544208 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:44.584465 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9516d11-eb64-4f2c-932c-be5068a63aa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:44.585089 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9516d11-eb64-4f2c-932c-be5068a63aa1 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:44.585373 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9516d11-eb64-4f2c-932c-be5068a63aa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:44.585651 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9516d11-eb64-4f2c-932c-be5068a63aa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:44.606488 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9516d11-eb64-4f2c-932c-be5068a63aa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:44.606488 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9516d11-eb64-4f2c-932c-be5068a63aa1 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:44.613082 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9649fe-3b18-4f7c-8a15-19ad12871a50 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:44.613610 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 962/1908] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:42:44.617511 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cbd4de-8f20-4ac4-937b-9ab8fe1bfb34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:44.618424 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbd4de-8f20-4ac4-937b-9ab8fe1bfb34 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:44.618813 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbd4de-8f20-4ac4-937b-9ab8fe1bfb34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:44.619438 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbd4de-8f20-4ac4-937b-9ab8fe1bfb34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:44.629554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cbd4de-8f20-4ac4-937b-9ab8fe1bfb34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:44.631020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cbd4de-8f20-4ac4-937b-9ab8fe1bfb34 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:44.641273 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 947/1909] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:42:45.277517 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cbd4de-8f20-4ac4-937b-9ab8fe1bfb34 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:45.278667 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9516d11-eb64-4f2c-932c-be5068a63aa1 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:45.282651 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a9516d11-eb64-4f2c-932c-be5068a63aa1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:45.283024 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-31cbd4de-8f20-4ac4-937b-9ab8fe1bfb34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:45.317249 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9516d11-eb64-4f2c-932c-be5068a63aa1 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:45.323764 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cbd4de-8f20-4ac4-937b-9ab8fe1bfb34 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:45.327249 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f934d1-aa22-4f25-afc3-85545caa0d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:45.330750 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f934d1-aa22-4f25-afc3-85545caa0d54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:45.330750 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f934d1-aa22-4f25-afc3-85545caa0d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:45.330750 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f934d1-aa22-4f25-afc3-85545caa0d54 admin admin] PATH_INFO: `/v3/projects/00fa5ee6d2034dde9813e4d0a141aec4` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:45.330750 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f934d1-aa22-4f25-afc3-85545caa0d54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00fa5ee6d2034dde9813e4d0a141aec4)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:45.332707 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f934d1-aa22-4f25-afc3-85545caa0d54 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:45.339539 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7a4b4-72ce-4a94-9d2f-a9dde25b079b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:45.340195 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7a4b4-72ce-4a94-9d2f-a9dde25b079b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:45.340475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7a4b4-72ce-4a94-9d2f-a9dde25b079b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:45.340766 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7a4b4-72ce-4a94-9d2f-a9dde25b079b admin admin] PATH_INFO: `/v3/projects/a0f0e852f53e451aa5c64e6f1a8b4125` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:45.341998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f7a4b4-72ce-4a94-9d2f-a9dde25b079b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0f0e852f53e451aa5c64e6f1a8b4125)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:45.343873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f7a4b4-72ce-4a94-9d2f-a9dde25b079b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:45.454089 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-19f934d1-aa22-4f25-afc3-85545caa0d54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00fa5ee6d2034dde9813e4d0a141aec4', 'request_id': 'req-19f934d1-aa22-4f25-afc3-85545caa0d54'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:45.459644 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-19f934d1-aa22-4f25-afc3-85545caa0d54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00fa5ee6d2034dde9813e4d0a141aec4', 'request_id': 'req-19f934d1-aa22-4f25-afc3-85545caa0d54'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:45.489650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d0f7a4b4-72ce-4a94-9d2f-a9dde25b079b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0f0e852f53e451aa5c64e6f1a8b4125', 'request_id': 'req-d0f7a4b4-72ce-4a94-9d2f-a9dde25b079b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:45.495992 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d0f7a4b4-72ce-4a94-9d2f-a9dde25b079b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0f0e852f53e451aa5c64e6f1a8b4125', 'request_id': 'req-d0f7a4b4-72ce-4a94-9d2f-a9dde25b079b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:45.551246 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-19f934d1-aa22-4f25-afc3-85545caa0d54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00fa5ee6d2034dde9813e4d0a141aec4', 'request_id': 'req-19f934d1-aa22-4f25-afc3-85545caa0d54'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:45.566045 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-19f934d1-aa22-4f25-afc3-85545caa0d54 admin admin] The token cache is being invalidate because project 00fa5ee6d2034dde9813e4d0a141aec4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:42:45.566502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-19f934d1-aa22-4f25-afc3-85545caa0d54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:45.567219 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-19f934d1-aa22-4f25-afc3-85545caa0d54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:45.568918 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 963/1910] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:42:45 2022] DELETE /identity/v3/projects/00fa5ee6d2034dde9813e4d0a141aec4 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:45.573660 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f934d1-aa22-4f25-afc3-85545caa0d54 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:45.575030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-19f934d1-aa22-4f25-afc3-85545caa0d54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:45.642428 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d0f7a4b4-72ce-4a94-9d2f-a9dde25b079b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0f0e852f53e451aa5c64e6f1a8b4125', 'request_id': 'req-d0f7a4b4-72ce-4a94-9d2f-a9dde25b079b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:45.645741 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f934d1-aa22-4f25-afc3-85545caa0d54 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:45.657089 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d0f7a4b4-72ce-4a94-9d2f-a9dde25b079b admin admin] The token cache is being invalidate because project a0f0e852f53e451aa5c64e6f1a8b4125 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:42:45.657089 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d0f7a4b4-72ce-4a94-9d2f-a9dde25b079b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:45.657235 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d0f7a4b4-72ce-4a94-9d2f-a9dde25b079b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:45.659121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 948/1911] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:42:45 2022] DELETE /identity/v3/projects/a0f0e852f53e451aa5c64e6f1a8b4125 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:45.662674 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7a4b4-72ce-4a94-9d2f-a9dde25b079b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:45.664405 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d0f7a4b4-72ce-4a94-9d2f-a9dde25b079b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:45.674527 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:45.714063 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:45.723325 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729d138-13ef-4123-b7d9-f0cb5a32bd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:45.724282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729d138-13ef-4123-b7d9-f0cb5a32bd67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:45.724573 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729d138-13ef-4123-b7d9-f0cb5a32bd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:45.724867 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729d138-13ef-4123-b7d9-f0cb5a32bd67 admin admin] PATH_INFO: `/v3/users/872d4715974f480199edb4a571441bc0` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:45.726004 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c729d138-13ef-4123-b7d9-f0cb5a32bd67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=872d4715974f480199edb4a571441bc0)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:45.728957 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c729d138-13ef-4123-b7d9-f0cb5a32bd67 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:45.731566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7a4b4-72ce-4a94-9d2f-a9dde25b079b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:45.736632 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aea686-9954-4d5f-a2ba-04c0d9b74614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:45.737267 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aea686-9954-4d5f-a2ba-04c0d9b74614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:45.737550 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aea686-9954-4d5f-a2ba-04c0d9b74614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:45.738059 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aea686-9954-4d5f-a2ba-04c0d9b74614 admin admin] PATH_INFO: `/v3/users/525deea5f74241ff87025f79faf2969c` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:45.739163 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aea686-9954-4d5f-a2ba-04c0d9b74614 admin admin] RBAC: Authorizing `identity:delete_user(user_id=525deea5f74241ff87025f79faf2969c)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:45.741115 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aea686-9954-4d5f-a2ba-04c0d9b74614 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:45.941871 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c729d138-13ef-4123-b7d9-f0cb5a32bd67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '872d4715974f480199edb4a571441bc0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:45.949735 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c729d138-13ef-4123-b7d9-f0cb5a32bd67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '872d4715974f480199edb4a571441bc0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:46.010448 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-78aea686-9954-4d5f-a2ba-04c0d9b74614 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '525deea5f74241ff87025f79faf2969c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:46.014548 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c729d138-13ef-4123-b7d9-f0cb5a32bd67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '872d4715974f480199edb4a571441bc0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:46.015219 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c729d138-13ef-4123-b7d9-f0cb5a32bd67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '872d4715974f480199edb4a571441bc0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:46.015821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c729d138-13ef-4123-b7d9-f0cb5a32bd67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '872d4715974f480199edb4a571441bc0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:46.033549 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 964/1912] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:42:45 2022] DELETE /identity/v3/users/872d4715974f480199edb4a571441bc0 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:42:46.082307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-78aea686-9954-4d5f-a2ba-04c0d9b74614 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '525deea5f74241ff87025f79faf2969c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:46.083098 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-78aea686-9954-4d5f-a2ba-04c0d9b74614 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '525deea5f74241ff87025f79faf2969c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:46.083744 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-78aea686-9954-4d5f-a2ba-04c0d9b74614 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '525deea5f74241ff87025f79faf2969c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:46.091949 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-78aea686-9954-4d5f-a2ba-04c0d9b74614 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '525deea5f74241ff87025f79faf2969c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:46.109756 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 949/1913] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:42:45 2022] DELETE /identity/v3/users/525deea5f74241ff87025f79faf2969c => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:42:46.518565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729d138-13ef-4123-b7d9-f0cb5a32bd67 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:46.520122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c729d138-13ef-4123-b7d9-f0cb5a32bd67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:46.580197 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aea686-9954-4d5f-a2ba-04c0d9b74614 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:46.582323 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-78aea686-9954-4d5f-a2ba-04c0d9b74614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:46.589342 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729d138-13ef-4123-b7d9-f0cb5a32bd67 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:46.661266 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14db6d-1f28-4764-8332-e6c1eb9bed14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:46.662009 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14db6d-1f28-4764-8332-e6c1eb9bed14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:46.662396 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14db6d-1f28-4764-8332-e6c1eb9bed14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:46.662778 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14db6d-1f28-4764-8332-e6c1eb9bed14 admin admin] PATH_INFO: `/v3/projects/81e028c37f334ce6bbcce6f6ba570c74` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:46.664076 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14db6d-1f28-4764-8332-e6c1eb9bed14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81e028c37f334ce6bbcce6f6ba570c74)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:46.666461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14db6d-1f28-4764-8332-e6c1eb9bed14 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:46.667518 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aea686-9954-4d5f-a2ba-04c0d9b74614 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:46.673087 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff607afc-727a-448f-b39a-bafab0554f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:46.674453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff607afc-727a-448f-b39a-bafab0554f91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:46.674738 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff607afc-727a-448f-b39a-bafab0554f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:46.675011 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff607afc-727a-448f-b39a-bafab0554f91 admin admin] PATH_INFO: `/v3/projects/67fcddbfd9364fdd858ea3d4574a46a5` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:46.678497 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff607afc-727a-448f-b39a-bafab0554f91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67fcddbfd9364fdd858ea3d4574a46a5)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:46.681094 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff607afc-727a-448f-b39a-bafab0554f91 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:46.705142 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:46.794248 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:46.811982 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ff607afc-727a-448f-b39a-bafab0554f91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67fcddbfd9364fdd858ea3d4574a46a5', 'request_id': 'req-ff607afc-727a-448f-b39a-bafab0554f91'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:46.813077 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1e14db6d-1f28-4764-8332-e6c1eb9bed14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81e028c37f334ce6bbcce6f6ba570c74', 'request_id': 'req-1e14db6d-1f28-4764-8332-e6c1eb9bed14'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:46.820259 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ff607afc-727a-448f-b39a-bafab0554f91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67fcddbfd9364fdd858ea3d4574a46a5', 'request_id': 'req-ff607afc-727a-448f-b39a-bafab0554f91'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:46.821101 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1e14db6d-1f28-4764-8332-e6c1eb9bed14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81e028c37f334ce6bbcce6f6ba570c74', 'request_id': 'req-1e14db6d-1f28-4764-8332-e6c1eb9bed14'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:46.904373 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ff607afc-727a-448f-b39a-bafab0554f91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67fcddbfd9364fdd858ea3d4574a46a5', 'request_id': 'req-ff607afc-727a-448f-b39a-bafab0554f91'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:46.918439 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ff607afc-727a-448f-b39a-bafab0554f91 admin admin] The token cache is being invalidate because project 67fcddbfd9364fdd858ea3d4574a46a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:42:46.918890 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ff607afc-727a-448f-b39a-bafab0554f91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:46.919607 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ff607afc-727a-448f-b39a-bafab0554f91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:46.924528 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 950/1914] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:42:46 2022] DELETE /identity/v3/projects/67fcddbfd9364fdd858ea3d4574a46a5 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:42:46.924528 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff607afc-727a-448f-b39a-bafab0554f91 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:46.924528 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ff607afc-727a-448f-b39a-bafab0554f91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:46.991512 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1e14db6d-1f28-4764-8332-e6c1eb9bed14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81e028c37f334ce6bbcce6f6ba570c74', 'request_id': 'req-1e14db6d-1f28-4764-8332-e6c1eb9bed14'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:47.014259 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff607afc-727a-448f-b39a-bafab0554f91 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:47.021400 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1e14db6d-1f28-4764-8332-e6c1eb9bed14 admin admin] The token cache is being invalidate because project 81e028c37f334ce6bbcce6f6ba570c74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:42:47.021719 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1e14db6d-1f28-4764-8332-e6c1eb9bed14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:47.022450 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1e14db6d-1f28-4764-8332-e6c1eb9bed14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:47.024709 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 965/1915] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:42:46 2022] DELETE /identity/v3/projects/81e028c37f334ce6bbcce6f6ba570c74 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:47.029802 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c77c6a-8014-4b3e-953e-ee2e0faa59b6 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:47.029802 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c77c6a-8014-4b3e-953e-ee2e0faa59b6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:47.029802 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c77c6a-8014-4b3e-953e-ee2e0faa59b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:47.052451 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-41c77c6a-8014-4b3e-953e-ee2e0faa59b6 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:47.053801 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-41c77c6a-8014-4b3e-953e-ee2e0faa59b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:47.086279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2c981-6268-42b8-8227-cf6e610b58e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:47.099411 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2c981-6268-42b8-8227-cf6e610b58e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:47.099411 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2c981-6268-42b8-8227-cf6e610b58e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:47.099411 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2c981-6268-42b8-8227-cf6e610b58e6 admin admin] PATH_INFO: `/v3/users/7f8d2e7209c44718a59383d4c2bcd855` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:47.100995 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d2c981-6268-42b8-8227-cf6e610b58e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f8d2e7209c44718a59383d4c2bcd855)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:47.103623 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d2c981-6268-42b8-8227-cf6e610b58e6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:47.141818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 966/1916] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:47.143423 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4a97e-3747-4f36-b20b-2b7d93fe6c1c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:47.143641 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4a97e-3747-4f36-b20b-2b7d93fe6c1c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:47.144102 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4a97e-3747-4f36-b20b-2b7d93fe6c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:47.166420 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-cce4a97e-3747-4f36-b20b-2b7d93fe6c1c None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:47.167583 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cce4a97e-3747-4f36-b20b-2b7d93fe6c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:47.175667 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 967/1917] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:47.176924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41877c-e69b-4da8-8039-120e050d4f3b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:47.177210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41877c-e69b-4da8-8039-120e050d4f3b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:47.177485 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41877c-e69b-4da8-8039-120e050d4f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:47.199631 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ec41877c-e69b-4da8-8039-120e050d4f3b None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:47.200973 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ec41877c-e69b-4da8-8039-120e050d4f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:47.209725 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 968/1918] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:47.210923 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802a95e-c46b-454b-a3d9-ed204f9ae755 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:47.211228 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802a95e-c46b-454b-a3d9-ed204f9ae755 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:47.211505 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802a95e-c46b-454b-a3d9-ed204f9ae755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:47.231011 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d802a95e-c46b-454b-a3d9-ed204f9ae755 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:47.232258 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d802a95e-c46b-454b-a3d9-ed204f9ae755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:47.240962 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 969/1919] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:47.241802 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d802a95e-c46b-454b-a3d9-ed204f9ae755 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:47.252012 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d802a95e-c46b-454b-a3d9-ed204f9ae755 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:47.260884 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ccdba-cac9-4f2e-8d6b-575266cabf91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:47.261490 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ccdba-cac9-4f2e-8d6b-575266cabf91 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:47.261766 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ccdba-cac9-4f2e-8d6b-575266cabf91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:47.262118 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ccdba-cac9-4f2e-8d6b-575266cabf91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:47.262716 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ccdba-cac9-4f2e-8d6b-575266cabf91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:42:47.263017 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ccdba-cac9-4f2e-8d6b-575266cabf91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:47.264779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ccdba-cac9-4f2e-8d6b-575266cabf91 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:47.281593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 970/1920] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:42:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:42:47.283189 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ccdba-cac9-4f2e-8d6b-575266cabf91 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:47.296300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ccdba-cac9-4f2e-8d6b-575266cabf91 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:47.301468 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea363a2-5b9b-4ff2-ae43-f4daf828632c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:47.302103 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea363a2-5b9b-4ff2-ae43-f4daf828632c admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:47.302512 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea363a2-5b9b-4ff2-ae43-f4daf828632c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:47.302788 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea363a2-5b9b-4ff2-ae43-f4daf828632c admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:47.303593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea363a2-5b9b-4ff2-ae43-f4daf828632c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:42:47.303943 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea363a2-5b9b-4ff2-ae43-f4daf828632c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:47.305521 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea363a2-5b9b-4ff2-ae43-f4daf828632c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:47.314068 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e6d2c981-6268-42b8-8227-cf6e610b58e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f8d2e7209c44718a59383d4c2bcd855'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:47.319386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 971/1921] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:42:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:42:47.320202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea363a2-5b9b-4ff2-ae43-f4daf828632c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:47.330272 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea363a2-5b9b-4ff2-ae43-f4daf828632c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:47.335710 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb5749-a8e1-4ff8-bfb6-3a1162f19b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:47.336288 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb5749-a8e1-4ff8-bfb6-3a1162f19b27 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:47.336499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb5749-a8e1-4ff8-bfb6-3a1162f19b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:47.336819 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb5749-a8e1-4ff8-bfb6-3a1162f19b27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:47.337476 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb5749-a8e1-4ff8-bfb6-3a1162f19b27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:47.339373 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb5749-a8e1-4ff8-bfb6-3a1162f19b27 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:47.396368 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e6d2c981-6268-42b8-8227-cf6e610b58e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f8d2e7209c44718a59383d4c2bcd855'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:47.396998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e6d2c981-6268-42b8-8227-cf6e610b58e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f8d2e7209c44718a59383d4c2bcd855'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:47.397629 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e6d2c981-6268-42b8-8227-cf6e610b58e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f8d2e7209c44718a59383d4c2bcd855'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:47.405609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e6d2c981-6268-42b8-8227-cf6e610b58e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f8d2e7209c44718a59383d4c2bcd855'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:47.407504 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:47.423889 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 951/1922] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:42:46 2022] DELETE /identity/v3/users/7f8d2e7209c44718a59383d4c2bcd855 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:47.424860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2c981-6268-42b8-8227-cf6e610b58e6 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:47.426321 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e6d2c981-6268-42b8-8227-cf6e610b58e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:47.461367 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 972/1923] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:42:47 2022] POST /identity/v3/projects => generated 375 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:42:47.463865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a667b11-04b8-485b-afb4-30f8e1a03050 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:47.464153 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a667b11-04b8-485b-afb4-30f8e1a03050 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:47.464426 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a667b11-04b8-485b-afb4-30f8e1a03050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:47.486429 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0a667b11-04b8-485b-afb4-30f8e1a03050 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:47.487698 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0a667b11-04b8-485b-afb4-30f8e1a03050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:47.505927 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2c981-6268-42b8-8227-cf6e610b58e6 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:47.529215 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:47.559770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 973/1924] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:47.560779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a667b11-04b8-485b-afb4-30f8e1a03050 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:47.564391 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0a667b11-04b8-485b-afb4-30f8e1a03050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:47.577659 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401b102-fa5f-404d-855a-0cfbdab82c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:47.577891 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a667b11-04b8-485b-afb4-30f8e1a03050 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:47.578409 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401b102-fa5f-404d-855a-0cfbdab82c69 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:47.578857 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401b102-fa5f-404d-855a-0cfbdab82c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:47.579158 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401b102-fa5f-404d-855a-0cfbdab82c69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:47.579835 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401b102-fa5f-404d-855a-0cfbdab82c69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:47.581871 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401b102-fa5f-404d-855a-0cfbdab82c69 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:47.584807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96873389-1155-4b89-8369-3db92ef211f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:47.585477 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96873389-1155-4b89-8369-3db92ef211f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:47.586085 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96873389-1155-4b89-8369-3db92ef211f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:47.586085 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96873389-1155-4b89-8369-3db92ef211f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:47.586956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96873389-1155-4b89-8369-3db92ef211f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:47.589953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96873389-1155-4b89-8369-3db92ef211f1 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:47.656269 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 952/1925] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:42:47 2022] POST /identity/v3/projects => generated 391 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:42:47.660282 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401b102-fa5f-404d-855a-0cfbdab82c69 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:47.661914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e401b102-fa5f-404d-855a-0cfbdab82c69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:47.676875 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 974/1926] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:42:47 2022] POST /identity/v3/users => generated 288 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:42:47.677537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96873389-1155-4b89-8369-3db92ef211f1 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:47.689966 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96873389-1155-4b89-8369-3db92ef211f1 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:47.729979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401b102-fa5f-404d-855a-0cfbdab82c69 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:47.733757 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd2662-90c2-407c-86e5-4f8740f12c63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:47.734371 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd2662-90c2-407c-86e5-4f8740f12c63 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:47.734654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd2662-90c2-407c-86e5-4f8740f12c63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:47.734932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd2662-90c2-407c-86e5-4f8740f12c63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:47.755108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd75c4cf-ac35-4cd8-a8ea-0fc9e639bc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:47.756047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd75c4cf-ac35-4cd8-a8ea-0fc9e639bc02 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:47.756167 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd75c4cf-ac35-4cd8-a8ea-0fc9e639bc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:47.756517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd75c4cf-ac35-4cd8-a8ea-0fc9e639bc02 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:47.757205 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd75c4cf-ac35-4cd8-a8ea-0fc9e639bc02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:47.759278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd75c4cf-ac35-4cd8-a8ea-0fc9e639bc02 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:47.800816 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd2662-90c2-407c-86e5-4f8740f12c63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:47.802523 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd2662-90c2-407c-86e5-4f8740f12c63 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:47.816063 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 953/1927] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:42:47.816908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd2662-90c2-407c-86e5-4f8740f12c63 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:47.826877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd2662-90c2-407c-86e5-4f8740f12c63 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:47.838853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09196cad-c0e3-49b8-b813-96183f0edc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:47.839811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09196cad-c0e3-49b8-b813-96183f0edc12 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:47.840415 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09196cad-c0e3-49b8-b813-96183f0edc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:47.840840 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09196cad-c0e3-49b8-b813-96183f0edc12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:47.841702 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09196cad-c0e3-49b8-b813-96183f0edc12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:47.848124 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09196cad-c0e3-49b8-b813-96183f0edc12 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:47.863702 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 954/1928] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:47 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:47.864703 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09196cad-c0e3-49b8-b813-96183f0edc12 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:47.865842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 975/1929] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:42:47 2022] POST /identity/v3/users => generated 296 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:42:47.869979 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd75c4cf-ac35-4cd8-a8ea-0fc9e639bc02 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:47.876788 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09196cad-c0e3-49b8-b813-96183f0edc12 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:47.882138 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac06137-f856-49aa-b7f4-cc0615d5b473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:47.883058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac06137-f856-49aa-b7f4-cc0615d5b473 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:47.883379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac06137-f856-49aa-b7f4-cc0615d5b473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:47.883666 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac06137-f856-49aa-b7f4-cc0615d5b473 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:47.884718 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac06137-f856-49aa-b7f4-cc0615d5b473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:47.886618 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac06137-f856-49aa-b7f4-cc0615d5b473 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:47.893219 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd75c4cf-ac35-4cd8-a8ea-0fc9e639bc02 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:47.893564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:47.900097 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93ecc5-1eea-49ab-b428-e23386b64916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:47.901671 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93ecc5-1eea-49ab-b428-e23386b64916 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:47.902107 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93ecc5-1eea-49ab-b428-e23386b64916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:47.902604 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93ecc5-1eea-49ab-b428-e23386b64916 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:47.903975 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf93ecc5-1eea-49ab-b428-e23386b64916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:47.904549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 955/1930] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:47.905987 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf93ecc5-1eea-49ab-b428-e23386b64916 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:47.916644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac06137-f856-49aa-b7f4-cc0615d5b473 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:47.921355 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 976/1931] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:47 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:47.925992 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93ecc5-1eea-49ab-b428-e23386b64916 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:47.940732 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac06137-f856-49aa-b7f4-cc0615d5b473 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:47.949649 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93ecc5-1eea-49ab-b428-e23386b64916 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:47.952160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1939097-0927-4216-9a19-5509cc0d19e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:47.954958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a106b52-fdd3-492a-a26b-cfedad3fa4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:47.955693 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1939097-0927-4216-9a19-5509cc0d19e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:47.955883 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a106b52-fdd3-492a-a26b-cfedad3fa4ba admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:47.956019 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1939097-0927-4216-9a19-5509cc0d19e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:47.956172 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a106b52-fdd3-492a-a26b-cfedad3fa4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:47.956275 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1939097-0927-4216-9a19-5509cc0d19e4 admin admin] PATH_INFO: `/v3/projects/b35b533969bf4d6b885ed34bf6d9a779/users/c45f298350ba47e99cfa15e77f8cd702/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:47.956450 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a106b52-fdd3-492a-a26b-cfedad3fa4ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:47.957100 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a106b52-fdd3-492a-a26b-cfedad3fa4ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:47.958768 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a106b52-fdd3-492a-a26b-cfedad3fa4ba admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:47.975861 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 977/1932] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:47 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:47.976867 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a106b52-fdd3-492a-a26b-cfedad3fa4ba admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:47.979481 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1939097-0927-4216-9a19-5509cc0d19e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b35b533969bf4d6b885ed34bf6d9a779, user_id=c45f298350ba47e99cfa15e77f8cd702, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:47.982366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1939097-0927-4216-9a19-5509cc0d19e4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:47.992431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a106b52-fdd3-492a-a26b-cfedad3fa4ba admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:47.998225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c41297-cb1a-48d3-b9a2-a45e84daa7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:47.999044 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c41297-cb1a-48d3-b9a2-a45e84daa7ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:47.999457 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c41297-cb1a-48d3-b9a2-a45e84daa7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:47.999834 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c41297-cb1a-48d3-b9a2-a45e84daa7ab admin admin] PATH_INFO: `/v3/projects/6c2ab18d87124d14b5e4622d636ff22c/users/235660d27a374cd49c057e03288369a3/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:48.018470 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c41297-cb1a-48d3-b9a2-a45e84daa7ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c2ab18d87124d14b5e4622d636ff22c, user_id=235660d27a374cd49c057e03288369a3, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:48.021325 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c41297-cb1a-48d3-b9a2-a45e84daa7ab admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:48.073490 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 956/1933] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:42:47 2022] PUT /identity/v3/projects/b35b533969bf4d6b885ed34bf6d9a779/users/c45f298350ba47e99cfa15e77f8cd702/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:48.083773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3f940-7aa8-4e9e-97cb-d76060523014 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:48.084194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3f940-7aa8-4e9e-97cb-d76060523014 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:48.084570 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3f940-7aa8-4e9e-97cb-d76060523014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:48.142598 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 978/1934] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:42:47 2022] PUT /identity/v3/projects/6c2ab18d87124d14b5e4622d636ff22c/users/235660d27a374cd49c057e03288369a3/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:48.142945 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-50b3f940-7aa8-4e9e-97cb-d76060523014 None None] MFA Rules not processed for user `c45f298350ba47e99cfa15e77f8cd702`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:48.142945 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-50b3f940-7aa8-4e9e-97cb-d76060523014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:48.145511 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1369a8-86c7-418e-bc9a-440764a845f8 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:48.145638 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1369a8-86c7-418e-bc9a-440764a845f8 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:48.145936 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1369a8-86c7-418e-bc9a-440764a845f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:48.191598 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-dc1369a8-86c7-418e-bc9a-440764a845f8 None None] MFA Rules not processed for user `235660d27a374cd49c057e03288369a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:48.193864 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-dc1369a8-86c7-418e-bc9a-440764a845f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:48.270949 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 957/1935] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:42:48.271948 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b3f940-7aa8-4e9e-97cb-d76060523014 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:48.281400 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:48.286876 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 979/1936] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:48 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:42:48.291525 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1369a8-86c7-418e-bc9a-440764a845f8 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:48.377105 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1369a8-86c7-418e-bc9a-440764a845f8 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:48.380797 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b3f940-7aa8-4e9e-97cb-d76060523014 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:48.382724 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab575853-0b93-4d91-8e6b-74aaf60c00e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:48.383642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab575853-0b93-4d91-8e6b-74aaf60c00e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:48.384117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab575853-0b93-4d91-8e6b-74aaf60c00e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:48.384493 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab575853-0b93-4d91-8e6b-74aaf60c00e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:48.385210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab575853-0b93-4d91-8e6b-74aaf60c00e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:48.385860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cba4b5-096e-4382-b4e0-2549fef06272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:48.386464 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cba4b5-096e-4382-b4e0-2549fef06272 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:48.386741 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cba4b5-096e-4382-b4e0-2549fef06272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:48.387015 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cba4b5-096e-4382-b4e0-2549fef06272 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:48.387731 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cba4b5-096e-4382-b4e0-2549fef06272 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:48.388460 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab575853-0b93-4d91-8e6b-74aaf60c00e8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:48.389640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cba4b5-096e-4382-b4e0-2549fef06272 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:48.408854 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:48.468723 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 980/1937] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:42:48 2022] POST /identity/v3/projects => generated 391 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:42:48.469289 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 958/1938] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:42:48 2022] POST /identity/v3/projects => generated 377 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:42:48.472805 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab575853-0b93-4d91-8e6b-74aaf60c00e8 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:48.473274 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cba4b5-096e-4382-b4e0-2549fef06272 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:48.474841 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ab575853-0b93-4d91-8e6b-74aaf60c00e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:48.544029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cba4b5-096e-4382-b4e0-2549fef06272 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:48.549637 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6489da-beb9-4089-96a5-c65c5af996f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:48.550314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6489da-beb9-4089-96a5-c65c5af996f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:48.550606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6489da-beb9-4089-96a5-c65c5af996f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:48.550878 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6489da-beb9-4089-96a5-c65c5af996f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:48.551541 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6489da-beb9-4089-96a5-c65c5af996f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:48.554072 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6489da-beb9-4089-96a5-c65c5af996f6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:48.554217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab575853-0b93-4d91-8e6b-74aaf60c00e8 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:48.560643 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381bbe4-0b0d-4e13-a469-2e1481e10fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:48.561412 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381bbe4-0b0d-4e13-a469-2e1481e10fc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:48.561816 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381bbe4-0b0d-4e13-a469-2e1481e10fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:48.562205 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381bbe4-0b0d-4e13-a469-2e1481e10fc0 admin admin] PATH_INFO: `/v3/projects/84b90313fb214c428bd651d7a68eae2a` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:48.563466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9381bbe4-0b0d-4e13-a469-2e1481e10fc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84b90313fb214c428bd651d7a68eae2a)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:48.565568 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9381bbe4-0b0d-4e13-a469-2e1481e10fc0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:48.633183 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 959/1939] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:42:48 2022] POST /identity/v3/users => generated 302 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:42:48.633183 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6489da-beb9-4089-96a5-c65c5af996f6 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:48.645221 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6489da-beb9-4089-96a5-c65c5af996f6 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:48.651386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a9ff0-7af1-4b6f-8b13-30ca8a52d671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:48.652053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a9ff0-7af1-4b6f-8b13-30ca8a52d671 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:48.652339 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a9ff0-7af1-4b6f-8b13-30ca8a52d671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:48.652610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a9ff0-7af1-4b6f-8b13-30ca8a52d671 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:48.653291 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3a9ff0-7af1-4b6f-8b13-30ca8a52d671 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:48.655254 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3a9ff0-7af1-4b6f-8b13-30ca8a52d671 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:48.702513 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9381bbe4-0b0d-4e13-a469-2e1481e10fc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84b90313fb214c428bd651d7a68eae2a', 'request_id': 'req-9381bbe4-0b0d-4e13-a469-2e1481e10fc0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:48.709365 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9381bbe4-0b0d-4e13-a469-2e1481e10fc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84b90313fb214c428bd651d7a68eae2a', 'request_id': 'req-9381bbe4-0b0d-4e13-a469-2e1481e10fc0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:48.743787 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 960/1940] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:42:48 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:42:48.743787 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a9ff0-7af1-4b6f-8b13-30ca8a52d671 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:48.752054 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a9ff0-7af1-4b6f-8b13-30ca8a52d671 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:48.796923 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9381bbe4-0b0d-4e13-a469-2e1481e10fc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84b90313fb214c428bd651d7a68eae2a', 'request_id': 'req-9381bbe4-0b0d-4e13-a469-2e1481e10fc0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:48.809465 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9381bbe4-0b0d-4e13-a469-2e1481e10fc0 admin admin] The token cache is being invalidate because project 84b90313fb214c428bd651d7a68eae2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:42:48.809757 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9381bbe4-0b0d-4e13-a469-2e1481e10fc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:48.810437 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9381bbe4-0b0d-4e13-a469-2e1481e10fc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:48.812381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 981/1941] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:42:48 2022] DELETE /identity/v3/projects/84b90313fb214c428bd651d7a68eae2a => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:48.816158 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381bbe4-0b0d-4e13-a469-2e1481e10fc0 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:48.817703 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9381bbe4-0b0d-4e13-a469-2e1481e10fc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:48.827168 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c8de1-f21f-45f1-a07d-f20ab5dfdc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:48.827789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c8de1-f21f-45f1-a07d-f20ab5dfdc5c admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:48.834457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c8de1-f21f-45f1-a07d-f20ab5dfdc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:48.834457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c8de1-f21f-45f1-a07d-f20ab5dfdc5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:48.834457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c8de1-f21f-45f1-a07d-f20ab5dfdc5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:48.834457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c8de1-f21f-45f1-a07d-f20ab5dfdc5c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:48.835442 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:48.841998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 961/1942] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:48 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:48.843145 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf0bca-283f-4a68-9848-bb386ae247ef None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:48.843460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf0bca-283f-4a68-9848-bb386ae247ef None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:48.843741 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf0bca-283f-4a68-9848-bb386ae247ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:48.866337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f8bf0bca-283f-4a68-9848-bb386ae247ef None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:48.867725 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f8bf0bca-283f-4a68-9848-bb386ae247ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:48.876245 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 962/1943] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:48.877022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf0bca-283f-4a68-9848-bb386ae247ef None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:48.878315 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f8bf0bca-283f-4a68-9848-bb386ae247ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:48.892979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf0bca-283f-4a68-9848-bb386ae247ef None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:48.898902 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f89ad-d5ee-4347-b249-8126d9cd6183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:48.900478 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f89ad-d5ee-4347-b249-8126d9cd6183 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:48.901075 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f89ad-d5ee-4347-b249-8126d9cd6183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:48.901589 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f89ad-d5ee-4347-b249-8126d9cd6183 admin admin] PATH_INFO: `/v3/projects/3c73b065bb93449dab92559fbe6ade8c/users/ec9f9e2c3ecf493e9245d4175c436af3/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:48.902599 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381bbe4-0b0d-4e13-a469-2e1481e10fc0 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:48.908710 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340bae3-cacf-4d57-bd07-ffab2218f412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:48.909909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340bae3-cacf-4d57-bd07-ffab2218f412 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:48.910271 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340bae3-cacf-4d57-bd07-ffab2218f412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:48.910608 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340bae3-cacf-4d57-bd07-ffab2218f412 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:48.911359 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d340bae3-cacf-4d57-bd07-ffab2218f412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:48.913213 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d340bae3-cacf-4d57-bd07-ffab2218f412 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:48.920217 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370f89ad-d5ee-4347-b249-8126d9cd6183 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c73b065bb93449dab92559fbe6ade8c, user_id=ec9f9e2c3ecf493e9245d4175c436af3, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:48.923882 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370f89ad-d5ee-4347-b249-8126d9cd6183 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:48.925004 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 982/1944] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:48 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:48.926746 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b6ace-44ae-407e-9ae7-7fceffc0aa57 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:48.927099 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b6ace-44ae-407e-9ae7-7fceffc0aa57 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:48.927470 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b6ace-44ae-407e-9ae7-7fceffc0aa57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:48.949837 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-241b6ace-44ae-407e-9ae7-7fceffc0aa57 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:48.951240 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-241b6ace-44ae-407e-9ae7-7fceffc0aa57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:48.966440 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 983/1945] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:48.966440 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b6ace-44ae-407e-9ae7-7fceffc0aa57 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:48.975894 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b6ace-44ae-407e-9ae7-7fceffc0aa57 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:48.980988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6dc8b1-1ebe-4100-a999-cde496a87cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:48.981606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6dc8b1-1ebe-4100-a999-cde496a87cca admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:48.981888 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6dc8b1-1ebe-4100-a999-cde496a87cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:48.982181 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6dc8b1-1ebe-4100-a999-cde496a87cca admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:48.982813 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6dc8b1-1ebe-4100-a999-cde496a87cca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:48.984574 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6dc8b1-1ebe-4100-a999-cde496a87cca admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:48.996540 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 963/1946] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:42:48 2022] PUT /identity/v3/projects/3c73b065bb93449dab92559fbe6ade8c/users/ec9f9e2c3ecf493e9245d4175c436af3/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:48.997830 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f89ad-d5ee-4347-b249-8126d9cd6183 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:48.999208 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 984/1947] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:49.001537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6dc8b1-1ebe-4100-a999-cde496a87cca admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:49.016320 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6dc8b1-1ebe-4100-a999-cde496a87cca admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:49.044594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:49.079981 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f89ad-d5ee-4347-b249-8126d9cd6183 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:49.085891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000532c-d875-4e0c-85c0-053b68dee894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:49.086508 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000532c-d875-4e0c-85c0-053b68dee894 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:49.086845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000532c-d875-4e0c-85c0-053b68dee894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:49.087338 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000532c-d875-4e0c-85c0-053b68dee894 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:49.088239 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000532c-d875-4e0c-85c0-053b68dee894 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:42:49.088546 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000532c-d875-4e0c-85c0-053b68dee894 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:49.090336 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000532c-d875-4e0c-85c0-053b68dee894 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:49.091588 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b991136-2a86-495c-8263-4e4ad7cd302d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:49.092420 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b991136-2a86-495c-8263-4e4ad7cd302d admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:49.093683 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b991136-2a86-495c-8263-4e4ad7cd302d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:49.094879 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b991136-2a86-495c-8263-4e4ad7cd302d admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:49.095566 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b991136-2a86-495c-8263-4e4ad7cd302d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:49.097169 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b991136-2a86-495c-8263-4e4ad7cd302d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:49.106782 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 964/1948] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:42:49.106782 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000532c-d875-4e0c-85c0-053b68dee894 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:49.110008 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 985/1949] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:49.111095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b991136-2a86-495c-8263-4e4ad7cd302d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:49.120069 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000532c-d875-4e0c-85c0-053b68dee894 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:49.125908 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b991136-2a86-495c-8263-4e4ad7cd302d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:49.128536 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087dd58-093c-45b6-801b-2ddf5e68c57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:49.129389 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087dd58-093c-45b6-801b-2ddf5e68c57b admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:49.129593 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087dd58-093c-45b6-801b-2ddf5e68c57b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:49.129881 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087dd58-093c-45b6-801b-2ddf5e68c57b admin admin] PATH_INFO: `/v3/projects/4d7d1ca0a8db42429b4e4bbae2dd401a/users/d89ab7e011284e179f9a202b54056999/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:49.133024 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39105621-a92a-4abf-ae53-16b2303c81ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:49.135629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39105621-a92a-4abf-ae53-16b2303c81ac admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:49.135964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39105621-a92a-4abf-ae53-16b2303c81ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:49.136233 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39105621-a92a-4abf-ae53-16b2303c81ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:49.144326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39105621-a92a-4abf-ae53-16b2303c81ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:49.144326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39105621-a92a-4abf-ae53-16b2303c81ac admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:49.154703 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4087dd58-093c-45b6-801b-2ddf5e68c57b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d7d1ca0a8db42429b4e4bbae2dd401a, user_id=d89ab7e011284e179f9a202b54056999, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:49.158964 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4087dd58-093c-45b6-801b-2ddf5e68c57b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:49.238456 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 986/1950] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:42:49 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:42:49.239586 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39105621-a92a-4abf-ae53-16b2303c81ac admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:49.254399 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 965/1951] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:42:49 2022] PUT /identity/v3/projects/4d7d1ca0a8db42429b4e4bbae2dd401a/users/d89ab7e011284e179f9a202b54056999/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:49.255486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087dd58-093c-45b6-801b-2ddf5e68c57b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:49.312381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39105621-a92a-4abf-ae53-16b2303c81ac admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:49.324047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:49.333294 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087dd58-093c-45b6-801b-2ddf5e68c57b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:49.340724 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a97e723-37bb-4bd7-8eaf-3b98591f5d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:49.341458 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97e723-37bb-4bd7-8eaf-3b98591f5d03 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:49.341910 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97e723-37bb-4bd7-8eaf-3b98591f5d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:49.342524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97e723-37bb-4bd7-8eaf-3b98591f5d03 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:49.343087 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a97e723-37bb-4bd7-8eaf-3b98591f5d03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:49.345162 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a97e723-37bb-4bd7-8eaf-3b98591f5d03 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:49.387035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d11085-7600-4776-9df0-9614e7ecc2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:49.388108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d11085-7600-4776-9df0-9614e7ecc2d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:49.388108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d11085-7600-4776-9df0-9614e7ecc2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:49.388283 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d11085-7600-4776-9df0-9614e7ecc2d2 admin admin] PATH_INFO: `/v3/projects/3c73b065bb93449dab92559fbe6ade8c/users/ec9f9e2c3ecf493e9245d4175c436af3/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:49.393780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d11085-7600-4776-9df0-9614e7ecc2d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c73b065bb93449dab92559fbe6ade8c, user_id=ec9f9e2c3ecf493e9245d4175c436af3, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:49.396819 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d11085-7600-4776-9df0-9614e7ecc2d2 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:49.409772 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 987/1952] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:42:49 2022] PUT /identity/v3/projects/3c73b065bb93449dab92559fbe6ade8c/users/ec9f9e2c3ecf493e9245d4175c436af3/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:49.419558 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005c450-5a29-4e51-86af-0db5aee0be51 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:49.419558 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005c450-5a29-4e51-86af-0db5aee0be51 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:49.419558 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005c450-5a29-4e51-86af-0db5aee0be51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:49.452315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b005c450-5a29-4e51-86af-0db5aee0be51 None None] MFA Rules not processed for user `d89ab7e011284e179f9a202b54056999`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:49.458152 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b005c450-5a29-4e51-86af-0db5aee0be51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:49.464482 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 966/1953] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:42:49 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:42:49.465928 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24228d-7379-4590-88c0-5d4953b481ec None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:49.466325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24228d-7379-4590-88c0-5d4953b481ec None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:49.466672 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24228d-7379-4590-88c0-5d4953b481ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:49.537790 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1e24228d-7379-4590-88c0-5d4953b481ec None None] MFA Rules not processed for user `ec9f9e2c3ecf493e9245d4175c436af3`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:49.539038 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1e24228d-7379-4590-88c0-5d4953b481ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:49.578207 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 988/1954] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:49.582779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b005c450-5a29-4e51-86af-0db5aee0be51 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:49.674204 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 967/1955] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:49 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:49.675058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24228d-7379-4590-88c0-5d4953b481ec None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:49.707376 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:49.710626 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b005c450-5a29-4e51-86af-0db5aee0be51 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:49.716963 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8905b-9047-41d1-be60-6c17f87a3c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:49.717749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8905b-9047-41d1-be60-6c17f87a3c64 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:49.718186 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8905b-9047-41d1-be60-6c17f87a3c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:49.718529 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8905b-9047-41d1-be60-6c17f87a3c64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:49.720016 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c8905b-9047-41d1-be60-6c17f87a3c64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:49.722263 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c8905b-9047-41d1-be60-6c17f87a3c64 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:49.736639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 989/1956] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:49.737729 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8905b-9047-41d1-be60-6c17f87a3c64 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:49.740559 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-52c8905b-9047-41d1-be60-6c17f87a3c64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:49.769081 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24228d-7379-4590-88c0-5d4953b481ec None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:49.775364 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be78f73-eee1-48a5-a434-5aa7a6d8e8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:49.776122 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be78f73-eee1-48a5-a434-5aa7a6d8e8ad admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:49.776500 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be78f73-eee1-48a5-a434-5aa7a6d8e8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:49.776925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be78f73-eee1-48a5-a434-5aa7a6d8e8ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:49.777628 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be78f73-eee1-48a5-a434-5aa7a6d8e8ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:49.778978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:49.779702 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be78f73-eee1-48a5-a434-5aa7a6d8e8ad admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:49.803594 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8905b-9047-41d1-be60-6c17f87a3c64 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:49.807964 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b091966-af7a-4925-b2f8-9e3f51464cce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:49.808581 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b091966-af7a-4925-b2f8-9e3f51464cce service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:49.808864 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b091966-af7a-4925-b2f8-9e3f51464cce service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:49.809896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b091966-af7a-4925-b2f8-9e3f51464cce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:49.818370 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b091966-af7a-4925-b2f8-9e3f51464cce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:49.819842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b091966-af7a-4925-b2f8-9e3f51464cce service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:49.830493 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 990/1957] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:49 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:42:49.831416 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b091966-af7a-4925-b2f8-9e3f51464cce service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:49.843285 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b091966-af7a-4925-b2f8-9e3f51464cce service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:49.846268 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708f5b7-1af7-4ec8-8527-bf648487d425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:49.846866 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708f5b7-1af7-4ec8-8527-bf648487d425 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:49.847136 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708f5b7-1af7-4ec8-8527-bf648487d425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:49.847432 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708f5b7-1af7-4ec8-8527-bf648487d425 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:49.848058 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708f5b7-1af7-4ec8-8527-bf648487d425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:49.849637 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708f5b7-1af7-4ec8-8527-bf648487d425 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:49.863364 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 991/1958] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:49.864474 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708f5b7-1af7-4ec8-8527-bf648487d425 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:49.876982 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 968/1959] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:42:49 2022] POST /identity/v3/projects => generated 375 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:42:49.877998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be78f73-eee1-48a5-a434-5aa7a6d8e8ad admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:49.880266 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708f5b7-1af7-4ec8-8527-bf648487d425 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:49.950456 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c5ceb-3fc2-49f3-a845-a2e3047e3aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:49.951086 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c5ceb-3fc2-49f3-a845-a2e3047e3aa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:49.951398 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c5ceb-3fc2-49f3-a845-a2e3047e3aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:49.951677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c5ceb-3fc2-49f3-a845-a2e3047e3aa2 admin admin] PATH_INFO: `/v3/projects/31b578119d744d0cb945cf93479e74c5/users/9575c52e5e534e9ab5bcc342e932cc41/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:49.953189 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be78f73-eee1-48a5-a434-5aa7a6d8e8ad admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:49.957894 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1771c-779b-41f4-9c75-5fc293b4adcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:49.958529 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1771c-779b-41f4-9c75-5fc293b4adcf admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:49.958806 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1771c-779b-41f4-9c75-5fc293b4adcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:49.959076 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1771c-779b-41f4-9c75-5fc293b4adcf admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:49.959792 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1771c-779b-41f4-9c75-5fc293b4adcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:49.961723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1771c-779b-41f4-9c75-5fc293b4adcf admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:49.966656 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c5ceb-3fc2-49f3-a845-a2e3047e3aa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31b578119d744d0cb945cf93479e74c5, user_id=9575c52e5e534e9ab5bcc342e932cc41, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:49.969460 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c5ceb-3fc2-49f3-a845-a2e3047e3aa2 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:50.075855 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 969/1960] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:42:49 2022] POST /identity/v3/users => generated 294 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:42:50.076737 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1771c-779b-41f4-9c75-5fc293b4adcf admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:50.078806 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f1b1771c-779b-41f4-9c75-5fc293b4adcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:50.094071 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 992/1961] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:42:49 2022] PUT /identity/v3/projects/31b578119d744d0cb945cf93479e74c5/users/9575c52e5e534e9ab5bcc342e932cc41/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:50.095548 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c5ceb-3fc2-49f3-a845-a2e3047e3aa2 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:50.169424 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1771c-779b-41f4-9c75-5fc293b4adcf admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:50.182178 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c5ceb-3fc2-49f3-a845-a2e3047e3aa2 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:50.187977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d1aac-021a-4588-8272-5cd8c6af2d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:50.188616 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d1aac-021a-4588-8272-5cd8c6af2d45 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:50.188961 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d1aac-021a-4588-8272-5cd8c6af2d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:50.189159 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d1aac-021a-4588-8272-5cd8c6af2d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:50.190525 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14d1aac-021a-4588-8272-5cd8c6af2d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:50.192187 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14d1aac-021a-4588-8272-5cd8c6af2d45 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:50.206711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 993/1962] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:50.206711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d84d3c-565e-4da3-a96c-d29e226aeaa1 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:50.206711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d84d3c-565e-4da3-a96c-d29e226aeaa1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:50.206711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d84d3c-565e-4da3-a96c-d29e226aeaa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:50.237823 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:50.238917 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a3d84d3c-565e-4da3-a96c-d29e226aeaa1 None None] MFA Rules not processed for user `9575c52e5e534e9ab5bcc342e932cc41`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:50.240386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a3d84d3c-565e-4da3-a96c-d29e226aeaa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:50.244980 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392e28e-405c-450d-9255-659c0c14911a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:50.245701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392e28e-405c-450d-9255-659c0c14911a service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:50.246043 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392e28e-405c-450d-9255-659c0c14911a service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:50.249201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392e28e-405c-450d-9255-659c0c14911a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:50.249201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b392e28e-405c-450d-9255-659c0c14911a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:50.292174 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:50.353384 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 994/1963] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:50.354127 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d84d3c-565e-4da3-a96c-d29e226aeaa1 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:50.368856 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d84d3c-565e-4da3-a96c-d29e226aeaa1 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:50.373089 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392e28e-405c-450d-9255-659c0c14911a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:50.374100 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194bd582-db5a-4f5e-aa4b-bffb324fb10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:50.374770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bd582-db5a-4f5e-aa4b-bffb324fb10c admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:50.375050 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bd582-db5a-4f5e-aa4b-bffb324fb10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:50.375582 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bd582-db5a-4f5e-aa4b-bffb324fb10c admin admin] PATH_INFO: `/v3/projects/7d57d9ef2e5d4c80a22d8082d32c0dc2/users/848bac99d20944dca78f86f666565bd1/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:50.377277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392e28e-405c-450d-9255-659c0c14911a service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:50.395094 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 970/1964] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:50 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:42:50.397695 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392e28e-405c-450d-9255-659c0c14911a service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:50.400936 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194bd582-db5a-4f5e-aa4b-bffb324fb10c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d57d9ef2e5d4c80a22d8082d32c0dc2, user_id=848bac99d20944dca78f86f666565bd1, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:50.404579 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194bd582-db5a-4f5e-aa4b-bffb324fb10c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:50.411362 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392e28e-405c-450d-9255-659c0c14911a service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:50.416893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c284e-b8a9-4a53-a28e-4d40455ae83c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:50.418208 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c284e-b8a9-4a53-a28e-4d40455ae83c service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:50.418501 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c284e-b8a9-4a53-a28e-4d40455ae83c service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:50.418780 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c284e-b8a9-4a53-a28e-4d40455ae83c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:50.429518 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c284e-b8a9-4a53-a28e-4d40455ae83c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:50.431014 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c284e-b8a9-4a53-a28e-4d40455ae83c service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:50.443628 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 971/1965] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:42:50.483753 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 995/1966] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:42:50 2022] PUT /identity/v3/projects/7d57d9ef2e5d4c80a22d8082d32c0dc2/users/848bac99d20944dca78f86f666565bd1/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:50.488439 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c284e-b8a9-4a53-a28e-4d40455ae83c service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:50.511355 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c284e-b8a9-4a53-a28e-4d40455ae83c service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:50.633597 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5aeaf-8ab5-4d96-83a9-1a465284bd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:50.635468 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5aeaf-8ab5-4d96-83a9-1a465284bd92 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:50.635777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5aeaf-8ab5-4d96-83a9-1a465284bd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:50.636054 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5aeaf-8ab5-4d96-83a9-1a465284bd92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:50.636706 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5aeaf-8ab5-4d96-83a9-1a465284bd92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:50.640015 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5aeaf-8ab5-4d96-83a9-1a465284bd92 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:50.652662 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 972/1967] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:50.656798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194bd582-db5a-4f5e-aa4b-bffb324fb10c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:50.672108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194bd582-db5a-4f5e-aa4b-bffb324fb10c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:50.687980 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8fbc22-c3c8-4d05-8a2d-24c89bce05f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:50.690008 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fbc22-c3c8-4d05-8a2d-24c89bce05f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:50.691501 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fbc22-c3c8-4d05-8a2d-24c89bce05f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:50.692861 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fbc22-c3c8-4d05-8a2d-24c89bce05f6 admin admin] PATH_INFO: `/v3/projects/7d57d9ef2e5d4c80a22d8082d32c0dc2/users/848bac99d20944dca78f86f666565bd1/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:50.696235 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8fbc22-c3c8-4d05-8a2d-24c89bce05f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d57d9ef2e5d4c80a22d8082d32c0dc2, user_id=848bac99d20944dca78f86f666565bd1, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:50.710976 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8fbc22-c3c8-4d05-8a2d-24c89bce05f6 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:50.736145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 996/1968] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:42:50 2022] PUT /identity/v3/projects/7d57d9ef2e5d4c80a22d8082d32c0dc2/users/848bac99d20944dca78f86f666565bd1/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:50.738613 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f4892-4102-4671-b592-32d8d5e1ae49 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:50.739066 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f4892-4102-4671-b592-32d8d5e1ae49 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:50.739158 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f4892-4102-4671-b592-32d8d5e1ae49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:50.773329 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7a4f4892-4102-4671-b592-32d8d5e1ae49 None None] MFA Rules not processed for user `848bac99d20944dca78f86f666565bd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:50.776310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7a4f4892-4102-4671-b592-32d8d5e1ae49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:50.888546 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 973/1969] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:50 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:51.098003 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a1630-9b03-40b4-b2f1-135f3df2ddee None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:51.098394 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a1630-9b03-40b4-b2f1-135f3df2ddee None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:51.098806 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a1630-9b03-40b4-b2f1-135f3df2ddee None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:51.100280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 997/1970] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 10 12:42:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:42:51.126908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb7b13-33fb-4d0f-8e4e-dffb3eaacced None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:51.127238 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb7b13-33fb-4d0f-8e4e-dffb3eaacced None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:51.127520 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb7b13-33fb-4d0f-8e4e-dffb3eaacced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:51.157114 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-83fb7b13-33fb-4d0f-8e4e-dffb3eaacced None None] MFA Rules not processed for user `3ff152e22374407eb72caf648a763107`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:51.158324 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-83fb7b13-33fb-4d0f-8e4e-dffb3eaacced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:51.163261 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:51.168850 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a1630-9b03-40b4-b2f1-135f3df2ddee None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:51.229354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 974/1971] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 10 12:42:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:42:51.230634 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a1630-9b03-40b4-b2f1-135f3df2ddee None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:51.237169 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51165c-9909-4a93-86ad-f7fd4a87c0d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:51.237801 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51165c-9909-4a93-86ad-f7fd4a87c0d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:51.238488 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51165c-9909-4a93-86ad-f7fd4a87c0d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:51.238488 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51165c-9909-4a93-86ad-f7fd4a87c0d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:51.240709 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bf51165c-9909-4a93-86ad-f7fd4a87c0d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:51.248073 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb7b13-33fb-4d0f-8e4e-dffb3eaacced None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:51.249826 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-83fb7b13-33fb-4d0f-8e4e-dffb3eaacced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:51.322839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51165c-9909-4a93-86ad-f7fd4a87c0d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:51.325042 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51165c-9909-4a93-86ad-f7fd4a87c0d2 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:51.340814 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 998/1972] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:42:51.343830 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51165c-9909-4a93-86ad-f7fd4a87c0d2 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:51.343937 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb7b13-33fb-4d0f-8e4e-dffb3eaacced None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:51.344035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bf51165c-9909-4a93-86ad-f7fd4a87c0d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:51.348224 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002519e-6dc0-4c9b-8f1a-90a4d4f64ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:51.349080 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002519e-6dc0-4c9b-8f1a-90a4d4f64ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:51.349559 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002519e-6dc0-4c9b-8f1a-90a4d4f64ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:51.349896 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002519e-6dc0-4c9b-8f1a-90a4d4f64ec4 admin admin] PATH_INFO: `/v3/users/51c598eb26614feaa0c31339b8135f08` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:51.351413 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f002519e-6dc0-4c9b-8f1a-90a4d4f64ec4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51c598eb26614feaa0c31339b8135f08)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:51.354230 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f002519e-6dc0-4c9b-8f1a-90a4d4f64ec4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:51.365955 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51165c-9909-4a93-86ad-f7fd4a87c0d2 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:51.381160 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da797e00-441c-4155-abad-4b842cf413e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff152e22374407eb72caf648a763107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:51.382002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da797e00-441c-4155-abad-4b842cf413e5 service placement] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:51.382489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da797e00-441c-4155-abad-4b842cf413e5 service placement] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:51.383193 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da797e00-441c-4155-abad-4b842cf413e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:51.398305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da797e00-441c-4155-abad-4b842cf413e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:51.400429 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da797e00-441c-4155-abad-4b842cf413e5 service placement] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:51.416103 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 999/1973] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:42:51.572483 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f002519e-6dc0-4c9b-8f1a-90a4d4f64ec4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51c598eb26614feaa0c31339b8135f08'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:51.692329 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f002519e-6dc0-4c9b-8f1a-90a4d4f64ec4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51c598eb26614feaa0c31339b8135f08'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:51.692999 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f002519e-6dc0-4c9b-8f1a-90a4d4f64ec4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51c598eb26614feaa0c31339b8135f08'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:51.693607 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f002519e-6dc0-4c9b-8f1a-90a4d4f64ec4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51c598eb26614feaa0c31339b8135f08'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:51.701767 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f002519e-6dc0-4c9b-8f1a-90a4d4f64ec4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51c598eb26614feaa0c31339b8135f08'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:51.719676 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 975/1974] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:42:51 2022] DELETE /identity/v3/users/51c598eb26614feaa0c31339b8135f08 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:51.961518 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da797e00-441c-4155-abad-4b842cf413e5 service placement] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:52.004995 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da797e00-441c-4155-abad-4b842cf413e5 service placement] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:52.057696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:52.084181 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23770770-5967-4b5e-9adf-63aa97cafc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:52.084181 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23770770-5967-4b5e-9adf-63aa97cafc3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:52.084181 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23770770-5967-4b5e-9adf-63aa97cafc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:52.084181 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23770770-5967-4b5e-9adf-63aa97cafc3d admin admin] PATH_INFO: `/v3/projects/fefe2d12cf334ecbad2001b327cb65e8` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:52.084639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23770770-5967-4b5e-9adf-63aa97cafc3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fefe2d12cf334ecbad2001b327cb65e8)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:52.084726 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23770770-5967-4b5e-9adf-63aa97cafc3d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:52.218558 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-23770770-5967-4b5e-9adf-63aa97cafc3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fefe2d12cf334ecbad2001b327cb65e8', 'request_id': 'req-23770770-5967-4b5e-9adf-63aa97cafc3d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:52.224714 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-23770770-5967-4b5e-9adf-63aa97cafc3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fefe2d12cf334ecbad2001b327cb65e8', 'request_id': 'req-23770770-5967-4b5e-9adf-63aa97cafc3d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:52.299449 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-23770770-5967-4b5e-9adf-63aa97cafc3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fefe2d12cf334ecbad2001b327cb65e8', 'request_id': 'req-23770770-5967-4b5e-9adf-63aa97cafc3d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:52.312045 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-23770770-5967-4b5e-9adf-63aa97cafc3d admin admin] The token cache is being invalidate because project fefe2d12cf334ecbad2001b327cb65e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:42:52.312339 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-23770770-5967-4b5e-9adf-63aa97cafc3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:52.312976 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-23770770-5967-4b5e-9adf-63aa97cafc3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:52.314471 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1000/1975] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:42:51 2022] DELETE /identity/v3/projects/fefe2d12cf334ecbad2001b327cb65e8 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:52.319560 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5438b-b6b4-419a-8253-0263cf902721 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:52.320033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5438b-b6b4-419a-8253-0263cf902721 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:52.320120 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5438b-b6b4-419a-8253-0263cf902721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:52.339592 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-35f5438b-b6b4-419a-8253-0263cf902721 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:52.340728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-35f5438b-b6b4-419a-8253-0263cf902721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:52.359637 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:52.403436 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 976/1976] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:52.411450 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160e68d-b334-4f3a-b8e3-3098472dd085 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:52.411742 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160e68d-b334-4f3a-b8e3-3098472dd085 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:52.412024 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160e68d-b334-4f3a-b8e3-3098472dd085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:52.433289 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-2160e68d-b334-4f3a-b8e3-3098472dd085 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:52.434509 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2160e68d-b334-4f3a-b8e3-3098472dd085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:52.443091 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1001/1977] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:52.447093 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5438b-b6b4-419a-8253-0263cf902721 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:52.458644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5438b-b6b4-419a-8253-0263cf902721 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:52.463624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b422d2-de20-4b87-98ee-c0c5a1f59270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:52.464577 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b422d2-de20-4b87-98ee-c0c5a1f59270 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:52.465067 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b422d2-de20-4b87-98ee-c0c5a1f59270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:52.465347 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b422d2-de20-4b87-98ee-c0c5a1f59270 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:52.466037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b422d2-de20-4b87-98ee-c0c5a1f59270 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:42:52.466335 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b422d2-de20-4b87-98ee-c0c5a1f59270 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:52.468220 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b422d2-de20-4b87-98ee-c0c5a1f59270 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:52.481090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 977/1978] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:42:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:42:52.485496 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160e68d-b334-4f3a-b8e3-3098472dd085 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:52.500863 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160e68d-b334-4f3a-b8e3-3098472dd085 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:52.505641 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce99c2d-47fe-4f65-8a5b-d70d7a2eae54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:52.506356 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce99c2d-47fe-4f65-8a5b-d70d7a2eae54 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:52.506639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce99c2d-47fe-4f65-8a5b-d70d7a2eae54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:52.506909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce99c2d-47fe-4f65-8a5b-d70d7a2eae54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:52.507599 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce99c2d-47fe-4f65-8a5b-d70d7a2eae54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:52.509824 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce99c2d-47fe-4f65-8a5b-d70d7a2eae54 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:52.667975 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1002/1979] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:42:52 2022] POST /identity/v3/projects => generated 403 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:42:52.672166 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b422d2-de20-4b87-98ee-c0c5a1f59270 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:52.683017 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b422d2-de20-4b87-98ee-c0c5a1f59270 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:52.741357 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc28ef-3fbb-4360-8b3a-67a007c57519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:52.742062 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc28ef-3fbb-4360-8b3a-67a007c57519 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:52.742344 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc28ef-3fbb-4360-8b3a-67a007c57519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:52.742616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc28ef-3fbb-4360-8b3a-67a007c57519 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:52.743297 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc28ef-3fbb-4360-8b3a-67a007c57519 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:52.745356 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc28ef-3fbb-4360-8b3a-67a007c57519 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:52.851727 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 978/1980] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:42:52 2022] POST /identity/v3/users => generated 302 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:42:52.852986 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce99c2d-47fe-4f65-8a5b-d70d7a2eae54 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:52.864570 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce99c2d-47fe-4f65-8a5b-d70d7a2eae54 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:52.869443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91faf39-38bc-4d48-aa57-69c3f76adaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:52.870195 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91faf39-38bc-4d48-aa57-69c3f76adaea admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:52.870482 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91faf39-38bc-4d48-aa57-69c3f76adaea admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:52.870763 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91faf39-38bc-4d48-aa57-69c3f76adaea admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:52.871481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91faf39-38bc-4d48-aa57-69c3f76adaea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:52.873072 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91faf39-38bc-4d48-aa57-69c3f76adaea admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:52.883131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1003/1981] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:52.887224 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc28ef-3fbb-4360-8b3a-67a007c57519 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:52.897680 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc28ef-3fbb-4360-8b3a-67a007c57519 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:52.903041 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace579b-0637-4842-957a-9d1aa84b1aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:52.903684 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace579b-0637-4842-957a-9d1aa84b1aba admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:52.903971 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace579b-0637-4842-957a-9d1aa84b1aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:52.904250 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace579b-0637-4842-957a-9d1aa84b1aba admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:52.904898 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace579b-0637-4842-957a-9d1aa84b1aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:52.906577 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace579b-0637-4842-957a-9d1aa84b1aba admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:52.918747 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 979/1982] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:52.923004 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91faf39-38bc-4d48-aa57-69c3f76adaea admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:52.934603 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91faf39-38bc-4d48-aa57-69c3f76adaea admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:52.940212 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e90d5-7aad-4a81-8506-70bd77567e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:52.947259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e90d5-7aad-4a81-8506-70bd77567e5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:52.947259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e90d5-7aad-4a81-8506-70bd77567e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:52.947259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e90d5-7aad-4a81-8506-70bd77567e5b admin admin] PATH_INFO: `/v3/projects/7144d0eb398940eeab4cd1f8a0ec4310/users/9abb52ab9ce24830a7c25baf7976cf45/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:52.964963 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e90d5-7aad-4a81-8506-70bd77567e5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7144d0eb398940eeab4cd1f8a0ec4310, user_id=9abb52ab9ce24830a7c25baf7976cf45, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:52.967668 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e90d5-7aad-4a81-8506-70bd77567e5b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:53.107105 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1004/1983] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:42:52 2022] PUT /identity/v3/projects/7144d0eb398940eeab4cd1f8a0ec4310/users/9abb52ab9ce24830a7c25baf7976cf45/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:53.112054 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481588f8-c652-4e54-b62a-2cb7c11bb66d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:53.112346 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481588f8-c652-4e54-b62a-2cb7c11bb66d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:53.112624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481588f8-c652-4e54-b62a-2cb7c11bb66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:53.135146 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-481588f8-c652-4e54-b62a-2cb7c11bb66d None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:53.138802 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-481588f8-c652-4e54-b62a-2cb7c11bb66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:53.209382 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 980/1984] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:53.221547 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e90d5-7aad-4a81-8506-70bd77567e5b admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:53.223418 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ff3e90d5-7aad-4a81-8506-70bd77567e5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:53.301843 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e90d5-7aad-4a81-8506-70bd77567e5b admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:53.306498 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a56b6-0772-4ccc-ac34-75d99ec691ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:53.307356 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a56b6-0772-4ccc-ac34-75d99ec691ce service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:53.307737 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a56b6-0772-4ccc-ac34-75d99ec691ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:53.308260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a56b6-0772-4ccc-ac34-75d99ec691ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:53.319010 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632a56b6-0772-4ccc-ac34-75d99ec691ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:53.320553 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632a56b6-0772-4ccc-ac34-75d99ec691ce service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:53.331815 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1005/1985] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:42:53.377294 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481588f8-c652-4e54-b62a-2cb7c11bb66d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:53.377294 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-481588f8-c652-4e54-b62a-2cb7c11bb66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:53.413495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:53.437070 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481588f8-c652-4e54-b62a-2cb7c11bb66d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:53.440839 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebbbab-34ec-4753-b69f-988459286596 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:53.445245 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebbbab-34ec-4753-b69f-988459286596 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:53.445245 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebbbab-34ec-4753-b69f-988459286596 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:53.445245 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebbbab-34ec-4753-b69f-988459286596 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:53.445245 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-97ebbbab-34ec-4753-b69f-988459286596 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:53.505036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ebbbab-34ec-4753-b69f-988459286596 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:53.506522 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ebbbab-34ec-4753-b69f-988459286596 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:53.516369 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 981/1986] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:42:54.734709 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9a616-c741-41e8-88db-c474a3e86890 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:54.735242 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9a616-c741-41e8-88db-c474a3e86890 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:54.735435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9a616-c741-41e8-88db-c474a3e86890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:54.750024 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:54.764850 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ebd9a616-c741-41e8-88db-c474a3e86890 None None] MFA Rules not processed for user `9abb52ab9ce24830a7c25baf7976cf45`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:54.766122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ebd9a616-c741-41e8-88db-c474a3e86890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:54.832084 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1006/1987] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:54 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:54.836675 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebbbab-34ec-4753-b69f-988459286596 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:54.848133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebbbab-34ec-4753-b69f-988459286596 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:54.853063 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23870744-0e87-40dd-a421-8abd72fb0324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:54.855909 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23870744-0e87-40dd-a421-8abd72fb0324 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:54.855909 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23870744-0e87-40dd-a421-8abd72fb0324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:54.855909 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23870744-0e87-40dd-a421-8abd72fb0324 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:54.855909 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23870744-0e87-40dd-a421-8abd72fb0324 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:54.857984 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23870744-0e87-40dd-a421-8abd72fb0324 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:54.938078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 982/1988] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:42:54 2022] POST /identity/v3/projects => generated 405 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:42:54.939308 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9a616-c741-41e8-88db-c474a3e86890 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:54.951915 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9a616-c741-41e8-88db-c474a3e86890 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:55.011505 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac20339-e2a2-4127-9dc4-ad80b4033347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:55.012128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac20339-e2a2-4127-9dc4-ad80b4033347 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:55.012411 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac20339-e2a2-4127-9dc4-ad80b4033347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:55.012686 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac20339-e2a2-4127-9dc4-ad80b4033347 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:55.013309 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac20339-e2a2-4127-9dc4-ad80b4033347 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:55.015260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac20339-e2a2-4127-9dc4-ad80b4033347 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:55.096668 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1007/1989] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:42:54 2022] POST /identity/v3/users => generated 309 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:42:55.098171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23870744-0e87-40dd-a421-8abd72fb0324 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:55.109034 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23870744-0e87-40dd-a421-8abd72fb0324 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:55.114719 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82152bb7-21d9-404b-bb2e-22885ef9b0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:55.119684 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82152bb7-21d9-404b-bb2e-22885ef9b0fb admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:55.119684 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82152bb7-21d9-404b-bb2e-22885ef9b0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:55.119684 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82152bb7-21d9-404b-bb2e-22885ef9b0fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:55.119684 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82152bb7-21d9-404b-bb2e-22885ef9b0fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:55.119684 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82152bb7-21d9-404b-bb2e-22885ef9b0fb admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:55.133693 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 983/1990] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:55.139465 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac20339-e2a2-4127-9dc4-ad80b4033347 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:55.149489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac20339-e2a2-4127-9dc4-ad80b4033347 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:55.154228 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b058e-59ff-4e32-a739-b028c8770085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:55.154888 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b058e-59ff-4e32-a739-b028c8770085 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:55.155185 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b058e-59ff-4e32-a739-b028c8770085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:55.155485 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b058e-59ff-4e32-a739-b028c8770085 admin admin] PATH_INFO: `/v3/projects/8561a7f1aa2b421096c89741a3741e3c/users/07ade4bf95fa49d9bf57a0c37c756d2d/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:55.175092 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b058e-59ff-4e32-a739-b028c8770085 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8561a7f1aa2b421096c89741a3741e3c, user_id=07ade4bf95fa49d9bf57a0c37c756d2d, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:55.177795 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b058e-59ff-4e32-a739-b028c8770085 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:55.283332 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1008/1991] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:42:55 2022] PUT /identity/v3/projects/8561a7f1aa2b421096c89741a3741e3c/users/07ade4bf95fa49d9bf57a0c37c756d2d/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:55.287256 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82152bb7-21d9-404b-bb2e-22885ef9b0fb admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:55.298104 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82152bb7-21d9-404b-bb2e-22885ef9b0fb admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:55.353930 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8abc5-16e7-424e-a991-0bdc58e7001d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:55.355052 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8abc5-16e7-424e-a991-0bdc58e7001d admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:55.355366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8abc5-16e7-424e-a991-0bdc58e7001d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:55.355642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8abc5-16e7-424e-a991-0bdc58e7001d admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:55.356279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e8abc5-16e7-424e-a991-0bdc58e7001d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:55.357889 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e8abc5-16e7-424e-a991-0bdc58e7001d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:55.367949 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 984/1992] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:55.372761 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b058e-59ff-4e32-a739-b028c8770085 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:55.382906 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b058e-59ff-4e32-a739-b028c8770085 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:55.387922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833cf4bb-ce25-4087-815d-e722eddb4f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:55.388563 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cf4bb-ce25-4087-815d-e722eddb4f43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:55.388840 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cf4bb-ce25-4087-815d-e722eddb4f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:55.389113 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cf4bb-ce25-4087-815d-e722eddb4f43 admin admin] PATH_INFO: `/v3/projects/8561a7f1aa2b421096c89741a3741e3c/users/07ade4bf95fa49d9bf57a0c37c756d2d/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:55.391010 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833cf4bb-ce25-4087-815d-e722eddb4f43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8561a7f1aa2b421096c89741a3741e3c, user_id=07ade4bf95fa49d9bf57a0c37c756d2d, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:55.393657 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833cf4bb-ce25-4087-815d-e722eddb4f43 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:55.403717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1009/1993] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:42:55 2022] PUT /identity/v3/projects/8561a7f1aa2b421096c89741a3741e3c/users/07ade4bf95fa49d9bf57a0c37c756d2d/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:56.719386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a47343-23d9-46d6-80d9-923de0f2e34c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:56.719927 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a47343-23d9-46d6-80d9-923de0f2e34c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:56.719927 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a47343-23d9-46d6-80d9-923de0f2e34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:56.755805 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f8a47343-23d9-46d6-80d9-923de0f2e34c None None] MFA Rules not processed for user `07ade4bf95fa49d9bf57a0c37c756d2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:56.757514 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f8a47343-23d9-46d6-80d9-923de0f2e34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:56.808875 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833cf4bb-ce25-4087-815d-e722eddb4f43 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:56.815522 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:56.853687 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 985/1994] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:56 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:56.878922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:56.887883 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833cf4bb-ce25-4087-815d-e722eddb4f43 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:56.892521 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a3835-b9c9-46d6-b3ce-384705d652f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:56.893364 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a3835-b9c9-46d6-b3ce-384705d652f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:56.893712 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a3835-b9c9-46d6-b3ce-384705d652f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:56.894038 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a3835-b9c9-46d6-b3ce-384705d652f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:56.897581 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-279a3835-b9c9-46d6-b3ce-384705d652f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:56.986611 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a3835-b9c9-46d6-b3ce-384705d652f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:56.988943 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a3835-b9c9-46d6-b3ce-384705d652f5 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:57.002701 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1010/1995] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:42:58.044709 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a47343-23d9-46d6-80d9-923de0f2e34c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:58.046232 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f8a47343-23d9-46d6-80d9-923de0f2e34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:58.132996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a47343-23d9-46d6-80d9-923de0f2e34c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:58.139336 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e150fa42-6ebe-4766-88bb-16e82b3bf501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:58.141921 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150fa42-6ebe-4766-88bb-16e82b3bf501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:58.141921 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150fa42-6ebe-4766-88bb-16e82b3bf501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:58.141921 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150fa42-6ebe-4766-88bb-16e82b3bf501 admin admin] PATH_INFO: `/v3/users/235660d27a374cd49c057e03288369a3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:58.141921 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e150fa42-6ebe-4766-88bb-16e82b3bf501 admin admin] RBAC: Authorizing `identity:delete_user(user_id=235660d27a374cd49c057e03288369a3)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:58.143755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e150fa42-6ebe-4766-88bb-16e82b3bf501 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:58.605993 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a3835-b9c9-46d6-b3ce-384705d652f5 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:58.619328 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a3835-b9c9-46d6-b3ce-384705d652f5 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:58.625383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f24f0f-4047-4900-8ebc-cca9306faff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:58.626125 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f24f0f-4047-4900-8ebc-cca9306faff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:58.626421 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f24f0f-4047-4900-8ebc-cca9306faff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:58.626549 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f24f0f-4047-4900-8ebc-cca9306faff8 admin admin] PATH_INFO: `/v3/users/9abb52ab9ce24830a7c25baf7976cf45` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:58.627708 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f24f0f-4047-4900-8ebc-cca9306faff8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9abb52ab9ce24830a7c25baf7976cf45)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:58.630031 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f24f0f-4047-4900-8ebc-cca9306faff8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:58.714747 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e150fa42-6ebe-4766-88bb-16e82b3bf501 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '235660d27a374cd49c057e03288369a3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:58.826309 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e150fa42-6ebe-4766-88bb-16e82b3bf501 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '235660d27a374cd49c057e03288369a3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:58.826933 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e150fa42-6ebe-4766-88bb-16e82b3bf501 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '235660d27a374cd49c057e03288369a3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:58.827731 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e150fa42-6ebe-4766-88bb-16e82b3bf501 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '235660d27a374cd49c057e03288369a3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:58.836577 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e150fa42-6ebe-4766-88bb-16e82b3bf501 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '235660d27a374cd49c057e03288369a3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:58.843973 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b7f24f0f-4047-4900-8ebc-cca9306faff8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9abb52ab9ce24830a7c25baf7976cf45'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:58.859501 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 986/1996] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:42:58 2022] DELETE /identity/v3/users/235660d27a374cd49c057e03288369a3 => generated 0 bytes in 815 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:58.864332 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b045b9e-2a82-4d75-bbf8-4774d14b7729 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:58.864694 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b045b9e-2a82-4d75-bbf8-4774d14b7729 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:58.865035 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b045b9e-2a82-4d75-bbf8-4774d14b7729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:58.866689 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b7f24f0f-4047-4900-8ebc-cca9306faff8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9abb52ab9ce24830a7c25baf7976cf45'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:58.884815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3b045b9e-2a82-4d75-bbf8-4774d14b7729 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:58.886635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3b045b9e-2a82-4d75-bbf8-4774d14b7729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:58.936711 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:58.948902 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 987/1997] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:42:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:42:58.958621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b045b9e-2a82-4d75-bbf8-4774d14b7729 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:58.960185 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3b045b9e-2a82-4d75-bbf8-4774d14b7729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:58.996505 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b7f24f0f-4047-4900-8ebc-cca9306faff8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9abb52ab9ce24830a7c25baf7976cf45'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:58.997237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b7f24f0f-4047-4900-8ebc-cca9306faff8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9abb52ab9ce24830a7c25baf7976cf45'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:58.997854 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b7f24f0f-4047-4900-8ebc-cca9306faff8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9abb52ab9ce24830a7c25baf7976cf45'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:59.022191 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b045b9e-2a82-4d75-bbf8-4774d14b7729 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:59.024544 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1011/1998] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:42:58 2022] DELETE /identity/v3/users/9abb52ab9ce24830a7c25baf7976cf45 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:59.027430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d97c6-5453-4d09-8875-a0c3bab80fe6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:59.028269 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d97c6-5453-4d09-8875-a0c3bab80fe6 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:59.028710 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d97c6-5453-4d09-8875-a0c3bab80fe6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:59.029021 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d97c6-5453-4d09-8875-a0c3bab80fe6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:59.030969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ab3d97c6-5453-4d09-8875-a0c3bab80fe6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:59.046053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3d97c6-5453-4d09-8875-a0c3bab80fe6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:59.048525 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3d97c6-5453-4d09-8875-a0c3bab80fe6 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:59.061306 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 988/1999] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:42:59.326085 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f24f0f-4047-4900-8ebc-cca9306faff8 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:59.327678 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b7f24f0f-4047-4900-8ebc-cca9306faff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:59.441578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f24f0f-4047-4900-8ebc-cca9306faff8 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:59.447033 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71a351-79e3-4974-bfaf-a726ecd2c80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:59.447943 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71a351-79e3-4974-bfaf-a726ecd2c80e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:59.449918 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71a351-79e3-4974-bfaf-a726ecd2c80e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:59.449918 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71a351-79e3-4974-bfaf-a726ecd2c80e admin admin] PATH_INFO: `/v3/projects/7144d0eb398940eeab4cd1f8a0ec4310` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:59.450835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a71a351-79e3-4974-bfaf-a726ecd2c80e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7144d0eb398940eeab4cd1f8a0ec4310)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:59.453302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a71a351-79e3-4974-bfaf-a726ecd2c80e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:59.467274 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:59.572634 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6a71a351-79e3-4974-bfaf-a726ecd2c80e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7144d0eb398940eeab4cd1f8a0ec4310', 'request_id': 'req-6a71a351-79e3-4974-bfaf-a726ecd2c80e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:59.577985 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6a71a351-79e3-4974-bfaf-a726ecd2c80e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7144d0eb398940eeab4cd1f8a0ec4310', 'request_id': 'req-6a71a351-79e3-4974-bfaf-a726ecd2c80e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:59.625045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d97c6-5453-4d09-8875-a0c3bab80fe6 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:59.626471 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ab3d97c6-5453-4d09-8875-a0c3bab80fe6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:59.638807 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d97c6-5453-4d09-8875-a0c3bab80fe6 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:59.669910 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6a71a351-79e3-4974-bfaf-a726ecd2c80e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7144d0eb398940eeab4cd1f8a0ec4310', 'request_id': 'req-6a71a351-79e3-4974-bfaf-a726ecd2c80e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:59.681892 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6a71a351-79e3-4974-bfaf-a726ecd2c80e admin admin] The token cache is being invalidate because project 7144d0eb398940eeab4cd1f8a0ec4310 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:42:59.682518 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6a71a351-79e3-4974-bfaf-a726ecd2c80e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:59.683206 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6a71a351-79e3-4974-bfaf-a726ecd2c80e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:59.684725 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1012/2000] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:42:59 2022] DELETE /identity/v3/projects/7144d0eb398940eeab4cd1f8a0ec4310 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:59.688682 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71a351-79e3-4974-bfaf-a726ecd2c80e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:59.690034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6a71a351-79e3-4974-bfaf-a726ecd2c80e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:59.699358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed220a09-67af-4941-b9a1-4a50923416fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:59.700002 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed220a09-67af-4941-b9a1-4a50923416fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:59.700288 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed220a09-67af-4941-b9a1-4a50923416fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:59.700564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed220a09-67af-4941-b9a1-4a50923416fe admin admin] PATH_INFO: `/v3/projects/6c2ab18d87124d14b5e4622d636ff22c` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:59.701690 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed220a09-67af-4941-b9a1-4a50923416fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c2ab18d87124d14b5e4622d636ff22c)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:59.703872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed220a09-67af-4941-b9a1-4a50923416fe admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:59.754550 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71a351-79e3-4974-bfaf-a726ecd2c80e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:59.759165 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ec67b-1677-45a0-a101-eb835fd3cb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:59.759823 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ec67b-1677-45a0-a101-eb835fd3cb03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:59.760100 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ec67b-1677-45a0-a101-eb835fd3cb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:59.760402 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ec67b-1677-45a0-a101-eb835fd3cb03 admin admin] PATH_INFO: `/v3/users/07ade4bf95fa49d9bf57a0c37c756d2d` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:59.761612 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ec67b-1677-45a0-a101-eb835fd3cb03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07ade4bf95fa49d9bf57a0c37c756d2d)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:59.763556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ec67b-1677-45a0-a101-eb835fd3cb03 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:59.848547 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ed220a09-67af-4941-b9a1-4a50923416fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c2ab18d87124d14b5e4622d636ff22c', 'request_id': 'req-ed220a09-67af-4941-b9a1-4a50923416fe'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:59.854047 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ed220a09-67af-4941-b9a1-4a50923416fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c2ab18d87124d14b5e4622d636ff22c', 'request_id': 'req-ed220a09-67af-4941-b9a1-4a50923416fe'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:59.922100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ed220a09-67af-4941-b9a1-4a50923416fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c2ab18d87124d14b5e4622d636ff22c', 'request_id': 'req-ed220a09-67af-4941-b9a1-4a50923416fe'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:59.933904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ed220a09-67af-4941-b9a1-4a50923416fe admin admin] The token cache is being invalidate because project 6c2ab18d87124d14b5e4622d636ff22c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:42:59.934194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ed220a09-67af-4941-b9a1-4a50923416fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:59.934845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ed220a09-67af-4941-b9a1-4a50923416fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:59.936671 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 989/2001] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:42:59 2022] DELETE /identity/v3/projects/6c2ab18d87124d14b5e4622d636ff22c => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:42:59.940282 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed220a09-67af-4941-b9a1-4a50923416fe admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:59.941676 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ed220a09-67af-4941-b9a1-4a50923416fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:59.966050 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-307ec67b-1677-45a0-a101-eb835fd3cb03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07ade4bf95fa49d9bf57a0c37c756d2d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:59.973352 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-307ec67b-1677-45a0-a101-eb835fd3cb03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07ade4bf95fa49d9bf57a0c37c756d2d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:00.009809 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed220a09-67af-4941-b9a1-4a50923416fe admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:00.015772 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12474d64-3b56-4411-b1d0-6a4474ad5ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:00.016852 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12474d64-3b56-4411-b1d0-6a4474ad5ddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:00.016852 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12474d64-3b56-4411-b1d0-6a4474ad5ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:00.017086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12474d64-3b56-4411-b1d0-6a4474ad5ddd admin admin] PATH_INFO: `/v3/users/ec9f9e2c3ecf493e9245d4175c436af3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:00.018059 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12474d64-3b56-4411-b1d0-6a4474ad5ddd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec9f9e2c3ecf493e9245d4175c436af3)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:00.020012 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12474d64-3b56-4411-b1d0-6a4474ad5ddd admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:00.037615 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:00.087649 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-307ec67b-1677-45a0-a101-eb835fd3cb03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07ade4bf95fa49d9bf57a0c37c756d2d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:00.088324 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-307ec67b-1677-45a0-a101-eb835fd3cb03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07ade4bf95fa49d9bf57a0c37c756d2d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:00.089255 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-307ec67b-1677-45a0-a101-eb835fd3cb03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07ade4bf95fa49d9bf57a0c37c756d2d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:00.107255 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1013/2002] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:42:59 2022] DELETE /identity/v3/users/07ade4bf95fa49d9bf57a0c37c756d2d => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:43:00.216128 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-12474d64-3b56-4411-b1d0-6a4474ad5ddd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec9f9e2c3ecf493e9245d4175c436af3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:00.288250 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-12474d64-3b56-4411-b1d0-6a4474ad5ddd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec9f9e2c3ecf493e9245d4175c436af3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:00.288898 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-12474d64-3b56-4411-b1d0-6a4474ad5ddd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec9f9e2c3ecf493e9245d4175c436af3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:00.290203 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-12474d64-3b56-4411-b1d0-6a4474ad5ddd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec9f9e2c3ecf493e9245d4175c436af3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:00.298399 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-12474d64-3b56-4411-b1d0-6a4474ad5ddd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec9f9e2c3ecf493e9245d4175c436af3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:00.317214 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 990/2003] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:42:59 2022] DELETE /identity/v3/users/ec9f9e2c3ecf493e9245d4175c436af3 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:00.384420 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ec67b-1677-45a0-a101-eb835fd3cb03 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:00.386896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-307ec67b-1677-45a0-a101-eb835fd3cb03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:00.412071 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:00.458684 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ec67b-1677-45a0-a101-eb835fd3cb03 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:00.527984 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d636e03-5500-408a-b5a9-5ec06745a52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:00.528790 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d636e03-5500-408a-b5a9-5ec06745a52c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:00.529121 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d636e03-5500-408a-b5a9-5ec06745a52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:00.529443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d636e03-5500-408a-b5a9-5ec06745a52c admin admin] PATH_INFO: `/v3/projects/8561a7f1aa2b421096c89741a3741e3c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:00.530695 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d636e03-5500-408a-b5a9-5ec06745a52c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8561a7f1aa2b421096c89741a3741e3c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:00.534754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d636e03-5500-408a-b5a9-5ec06745a52c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:00.690387 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3d636e03-5500-408a-b5a9-5ec06745a52c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8561a7f1aa2b421096c89741a3741e3c', 'request_id': 'req-3d636e03-5500-408a-b5a9-5ec06745a52c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:00.707877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3d636e03-5500-408a-b5a9-5ec06745a52c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8561a7f1aa2b421096c89741a3741e3c', 'request_id': 'req-3d636e03-5500-408a-b5a9-5ec06745a52c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:00.819311 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3d636e03-5500-408a-b5a9-5ec06745a52c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8561a7f1aa2b421096c89741a3741e3c', 'request_id': 'req-3d636e03-5500-408a-b5a9-5ec06745a52c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:00.832004 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3d636e03-5500-408a-b5a9-5ec06745a52c admin admin] The token cache is being invalidate because project 8561a7f1aa2b421096c89741a3741e3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:43:00.832295 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3d636e03-5500-408a-b5a9-5ec06745a52c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:00.832945 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3d636e03-5500-408a-b5a9-5ec06745a52c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:00.834447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1014/2004] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:43:00 2022] DELETE /identity/v3/projects/8561a7f1aa2b421096c89741a3741e3c => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:00.840081 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74f968-3ad4-44d3-b1c6-16a78d974fd1 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:00.840354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74f968-3ad4-44d3-b1c6-16a78d974fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:00.840678 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74f968-3ad4-44d3-b1c6-16a78d974fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:00.859765 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3b74f968-3ad4-44d3-b1c6-16a78d974fd1 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:00.861240 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3b74f968-3ad4-44d3-b1c6-16a78d974fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:00.874124 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d636e03-5500-408a-b5a9-5ec06745a52c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:00.875664 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3d636e03-5500-408a-b5a9-5ec06745a52c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:00.949725 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 991/2005] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:00.957530 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef30ff-f793-4e7e-8fc7-c467dbd455a6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:00.957820 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef30ff-f793-4e7e-8fc7-c467dbd455a6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:00.958116 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef30ff-f793-4e7e-8fc7-c467dbd455a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:00.958864 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d636e03-5500-408a-b5a9-5ec06745a52c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:00.964354 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d81378-d244-480d-80a4-b05688f2712a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:00.964940 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d81378-d244-480d-80a4-b05688f2712a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:00.965357 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d81378-d244-480d-80a4-b05688f2712a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:00.965715 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d81378-d244-480d-80a4-b05688f2712a admin admin] PATH_INFO: `/v3/projects/3c73b065bb93449dab92559fbe6ade8c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:00.968172 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d81378-d244-480d-80a4-b05688f2712a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c73b065bb93449dab92559fbe6ade8c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:00.970103 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d81378-d244-480d-80a4-b05688f2712a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:00.978383 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c6ef30ff-f793-4e7e-8fc7-c467dbd455a6 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:00.979953 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c6ef30ff-f793-4e7e-8fc7-c467dbd455a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:00.988449 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 992/2006] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:00.992887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef30ff-f793-4e7e-8fc7-c467dbd455a6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:01.002654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef30ff-f793-4e7e-8fc7-c467dbd455a6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:01.007267 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f4150c-c507-4bbe-9954-4472aeec1970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:01.007910 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4150c-c507-4bbe-9954-4472aeec1970 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:01.008510 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4150c-c507-4bbe-9954-4472aeec1970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:01.008716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4150c-c507-4bbe-9954-4472aeec1970 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:01.009357 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4150c-c507-4bbe-9954-4472aeec1970 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:43:01.009662 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4150c-c507-4bbe-9954-4472aeec1970 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:01.011918 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4150c-c507-4bbe-9954-4472aeec1970 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:01.024058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 993/2007] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:43:01.028451 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f4150c-c507-4bbe-9954-4472aeec1970 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:01.038090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f4150c-c507-4bbe-9954-4472aeec1970 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:01.043022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ce1af-f3ee-4d3a-b3cb-a80aaa34fd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:01.043662 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ce1af-f3ee-4d3a-b3cb-a80aaa34fd9d admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:01.043947 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ce1af-f3ee-4d3a-b3cb-a80aaa34fd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:01.044225 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ce1af-f3ee-4d3a-b3cb-a80aaa34fd9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:01.044843 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ce1af-f3ee-4d3a-b3cb-a80aaa34fd9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:01.046724 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ce1af-f3ee-4d3a-b3cb-a80aaa34fd9d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:01.121859 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-83d81378-d244-480d-80a4-b05688f2712a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c73b065bb93449dab92559fbe6ade8c', 'request_id': 'req-83d81378-d244-480d-80a4-b05688f2712a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:01.128171 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-83d81378-d244-480d-80a4-b05688f2712a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c73b065bb93449dab92559fbe6ade8c', 'request_id': 'req-83d81378-d244-480d-80a4-b05688f2712a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:01.131768 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 994/2008] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:43:01 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:43:01.136552 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ce1af-f3ee-4d3a-b3cb-a80aaa34fd9d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:01.149329 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:01.207082 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ce1af-f3ee-4d3a-b3cb-a80aaa34fd9d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:01.212013 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7833cbe-e1cd-4190-bbc5-a3e7c1977d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:01.212621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7833cbe-e1cd-4190-bbc5-a3e7c1977d5a admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:01.212903 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7833cbe-e1cd-4190-bbc5-a3e7c1977d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:01.213174 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7833cbe-e1cd-4190-bbc5-a3e7c1977d5a admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:01.214252 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7833cbe-e1cd-4190-bbc5-a3e7c1977d5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:01.216524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7833cbe-e1cd-4190-bbc5-a3e7c1977d5a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:01.219325 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-83d81378-d244-480d-80a4-b05688f2712a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c73b065bb93449dab92559fbe6ade8c', 'request_id': 'req-83d81378-d244-480d-80a4-b05688f2712a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:01.232250 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-83d81378-d244-480d-80a4-b05688f2712a admin admin] The token cache is being invalidate because project 3c73b065bb93449dab92559fbe6ade8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:43:01.232547 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-83d81378-d244-480d-80a4-b05688f2712a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:01.233302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-83d81378-d244-480d-80a4-b05688f2712a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:01.234814 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1015/2009] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:43:00 2022] DELETE /identity/v3/projects/3c73b065bb93449dab92559fbe6ade8c => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:01.239569 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c1d58-30dd-4f3c-90ce-e5adef6e1aac None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:01.239878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c1d58-30dd-4f3c-90ce-e5adef6e1aac None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:01.240156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c1d58-30dd-4f3c-90ce-e5adef6e1aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:01.260306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0d6c1d58-30dd-4f3c-90ce-e5adef6e1aac None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:01.262237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0d6c1d58-30dd-4f3c-90ce-e5adef6e1aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:01.270304 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1016/2010] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:43:01.278281 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a123df9-d53b-46e0-9567-3d0c9c45d42c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:01.278556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a123df9-d53b-46e0-9567-3d0c9c45d42c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:01.278830 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a123df9-d53b-46e0-9567-3d0c9c45d42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:01.298197 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9a123df9-d53b-46e0-9567-3d0c9c45d42c None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:01.299437 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9a123df9-d53b-46e0-9567-3d0c9c45d42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:01.308369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1017/2011] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:43:01.312573 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a123df9-d53b-46e0-9567-3d0c9c45d42c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:01.317227 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 995/2012] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:43:01 2022] POST /identity/v3/users => generated 283 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:43:01.321112 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7833cbe-e1cd-4190-bbc5-a3e7c1977d5a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:01.326056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a123df9-d53b-46e0-9567-3d0c9c45d42c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:01.326191 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d7833cbe-e1cd-4190-bbc5-a3e7c1977d5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:01.327681 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d7b44-ef18-4968-8857-f407c6d7a781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:01.328281 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d7b44-ef18-4968-8857-f407c6d7a781 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:01.328583 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d7b44-ef18-4968-8857-f407c6d7a781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:01.328857 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d7b44-ef18-4968-8857-f407c6d7a781 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:01.329489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782d7b44-ef18-4968-8857-f407c6d7a781 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:43:01.329790 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782d7b44-ef18-4968-8857-f407c6d7a781 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:01.331397 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782d7b44-ef18-4968-8857-f407c6d7a781 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:01.335064 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7833cbe-e1cd-4190-bbc5-a3e7c1977d5a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:01.339838 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd7861-23fc-46ce-b7ed-5e4d464ce3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:01.340437 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd7861-23fc-46ce-b7ed-5e4d464ce3e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:01.340716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd7861-23fc-46ce-b7ed-5e4d464ce3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:01.340989 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd7861-23fc-46ce-b7ed-5e4d464ce3e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:01.341632 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fd7861-23fc-46ce-b7ed-5e4d464ce3e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:01.343241 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fd7861-23fc-46ce-b7ed-5e4d464ce3e3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:01.346034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1018/2013] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:43:01.347927 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d7b44-ef18-4968-8857-f407c6d7a781 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:01.349557 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:01.359069 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 996/2014] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:43:01.361267 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d7b44-ef18-4968-8857-f407c6d7a781 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:01.363214 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd7861-23fc-46ce-b7ed-5e4d464ce3e3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:01.367376 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1cea55-8fa6-4527-aae0-ee5390150806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:01.367985 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cea55-8fa6-4527-aae0-ee5390150806 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:01.368262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cea55-8fa6-4527-aae0-ee5390150806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:01.368539 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cea55-8fa6-4527-aae0-ee5390150806 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:01.369209 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1cea55-8fa6-4527-aae0-ee5390150806 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:01.372014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1cea55-8fa6-4527-aae0-ee5390150806 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:01.375169 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd7861-23fc-46ce-b7ed-5e4d464ce3e3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:01.380041 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a067ef-c6c0-485b-bea4-7e1a0d7aafda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:01.380713 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a067ef-c6c0-485b-bea4-7e1a0d7aafda admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:01.381045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a067ef-c6c0-485b-bea4-7e1a0d7aafda admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:01.381344 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a067ef-c6c0-485b-bea4-7e1a0d7aafda admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:01.381977 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a067ef-c6c0-485b-bea4-7e1a0d7aafda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:01.384315 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a067ef-c6c0-485b-bea4-7e1a0d7aafda admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:01.395077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 997/2015] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:43:01.399246 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a067ef-c6c0-485b-bea4-7e1a0d7aafda admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:01.409130 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a067ef-c6c0-485b-bea4-7e1a0d7aafda admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:01.414164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7efa37-8d46-4074-aaee-d59947b4980c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:01.414808 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7efa37-8d46-4074-aaee-d59947b4980c admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:01.415088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7efa37-8d46-4074-aaee-d59947b4980c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:01.415405 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7efa37-8d46-4074-aaee-d59947b4980c admin admin] PATH_INFO: `/v3/projects/639a245c5a734323bd99f6eb2738a844/users/2770577a04484dfe965458658b9366c7/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:01.430562 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7efa37-8d46-4074-aaee-d59947b4980c admin admin] RBAC: Authorizing `identity:create_grant(project_id=639a245c5a734323bd99f6eb2738a844, user_id=2770577a04484dfe965458658b9366c7, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:01.434131 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7efa37-8d46-4074-aaee-d59947b4980c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:01.508371 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1019/2016] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:43:01 2022] POST /identity/v3/projects => generated 401 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:43:01.512511 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1cea55-8fa6-4527-aae0-ee5390150806 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:01.540789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 998/2017] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:43:01 2022] PUT /identity/v3/projects/639a245c5a734323bd99f6eb2738a844/users/2770577a04484dfe965458658b9366c7/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:43:01.542936 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344cb269-9839-49f7-8b9f-0e602afd487f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:01.543417 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344cb269-9839-49f7-8b9f-0e602afd487f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:01.543684 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344cb269-9839-49f7-8b9f-0e602afd487f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:01.575590 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-344cb269-9839-49f7-8b9f-0e602afd487f None None] MFA Rules not processed for user `2770577a04484dfe965458658b9366c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:01.576822 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-344cb269-9839-49f7-8b9f-0e602afd487f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:01.580791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1cea55-8fa6-4527-aae0-ee5390150806 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:01.641960 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5b01b-b352-420a-a7ba-8f4f35461acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:01.642494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5b01b-b352-420a-a7ba-8f4f35461acb admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:01.642770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5b01b-b352-420a-a7ba-8f4f35461acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:01.643039 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5b01b-b352-420a-a7ba-8f4f35461acb admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:01.643697 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5b01b-b352-420a-a7ba-8f4f35461acb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:01.644102 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 999/2018] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:01.653769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344cb269-9839-49f7-8b9f-0e602afd487f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:01.653881 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5b01b-b352-420a-a7ba-8f4f35461acb admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:01.653970 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-344cb269-9839-49f7-8b9f-0e602afd487f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:01.660643 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344cb269-9839-49f7-8b9f-0e602afd487f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:01.665311 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d754e6-8220-4bfb-a879-291d24cb2d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:01.665968 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d754e6-8220-4bfb-a879-291d24cb2d11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:01.666252 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d754e6-8220-4bfb-a879-291d24cb2d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:01.666529 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d754e6-8220-4bfb-a879-291d24cb2d11 admin admin] PATH_INFO: `/v3/users/9575c52e5e534e9ab5bcc342e932cc41` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:01.668462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d754e6-8220-4bfb-a879-291d24cb2d11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9575c52e5e534e9ab5bcc342e932cc41)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:01.670405 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d754e6-8220-4bfb-a879-291d24cb2d11 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:01.737654 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1020/2019] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:43:01 2022] POST /identity/v3/users => generated 301 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:43:01.737654 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5b01b-b352-420a-a7ba-8f4f35461acb admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:01.747298 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5b01b-b352-420a-a7ba-8f4f35461acb admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:01.752002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc151fa-36c9-4728-a913-1a5fd432a993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:01.752634 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc151fa-36c9-4728-a913-1a5fd432a993 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:01.753156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc151fa-36c9-4728-a913-1a5fd432a993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:01.753463 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc151fa-36c9-4728-a913-1a5fd432a993 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:01.754079 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc151fa-36c9-4728-a913-1a5fd432a993 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:01.756246 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc151fa-36c9-4728-a913-1a5fd432a993 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:01.882573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:01.884783 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1021/2020] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:43:01 2022] POST /identity/v3/projects => generated 363 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:43:01.889330 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc151fa-36c9-4728-a913-1a5fd432a993 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:01.896891 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:01.900903 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a5d754e6-8220-4bfb-a879-291d24cb2d11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9575c52e5e534e9ab5bcc342e932cc41'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:01.954236 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc151fa-36c9-4728-a913-1a5fd432a993 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:01.959036 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff08ef-59fc-436a-9627-94bd17f60cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:01.959672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff08ef-59fc-436a-9627-94bd17f60cca admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:01.959950 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff08ef-59fc-436a-9627-94bd17f60cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:01.960292 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff08ef-59fc-436a-9627-94bd17f60cca admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:01.961074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff08ef-59fc-436a-9627-94bd17f60cca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:01.962692 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff08ef-59fc-436a-9627-94bd17f60cca admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:01.972447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a5d754e6-8220-4bfb-a879-291d24cb2d11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9575c52e5e534e9ab5bcc342e932cc41'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:01.973455 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a5d754e6-8220-4bfb-a879-291d24cb2d11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9575c52e5e534e9ab5bcc342e932cc41'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:01.979624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a5d754e6-8220-4bfb-a879-291d24cb2d11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9575c52e5e534e9ab5bcc342e932cc41'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:02.002137 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a5d754e6-8220-4bfb-a879-291d24cb2d11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9575c52e5e534e9ab5bcc342e932cc41'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:02.002237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1022/2021] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:43:02.002237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff08ef-59fc-436a-9627-94bd17f60cca admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:02.002237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a3ff08ef-59fc-436a-9627-94bd17f60cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:02.021774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff08ef-59fc-436a-9627-94bd17f60cca admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:02.025382 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1000/2022] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:43:01 2022] DELETE /identity/v3/users/9575c52e5e534e9ab5bcc342e932cc41 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:02.026391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d754e6-8220-4bfb-a879-291d24cb2d11 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:02.028012 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a5d754e6-8220-4bfb-a879-291d24cb2d11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:02.033338 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c594e40-0f15-4f9d-ab92-2f789f917316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:02.034487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c594e40-0f15-4f9d-ab92-2f789f917316 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:02.034931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c594e40-0f15-4f9d-ab92-2f789f917316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:02.035266 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c594e40-0f15-4f9d-ab92-2f789f917316 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:02.039740 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c594e40-0f15-4f9d-ab92-2f789f917316 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:02.039740 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c594e40-0f15-4f9d-ab92-2f789f917316 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:02.101603 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d754e6-8220-4bfb-a879-291d24cb2d11 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:02.107313 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6aa93-aec3-4f75-a355-93115ac27914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:02.107923 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6aa93-aec3-4f75-a355-93115ac27914 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:02.108202 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6aa93-aec3-4f75-a355-93115ac27914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:02.108496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6aa93-aec3-4f75-a355-93115ac27914 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:02.109226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6aa93-aec3-4f75-a355-93115ac27914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:02.110806 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6aa93-aec3-4f75-a355-93115ac27914 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:02.122129 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1001/2023] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:43:02.123382 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35407887-5ab4-4e0e-aad5-7450f0077877 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:02.123669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35407887-5ab4-4e0e-aad5-7450f0077877 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:02.123944 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35407887-5ab4-4e0e-aad5-7450f0077877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:02.129202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1023/2024] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:43:01 2022] POST /identity/v3/users => generated 288 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:43:02.130915 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c594e40-0f15-4f9d-ab92-2f789f917316 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:02.134497 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0c594e40-0f15-4f9d-ab92-2f789f917316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:02.145299 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-35407887-5ab4-4e0e-aad5-7450f0077877 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:02.147929 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c594e40-0f15-4f9d-ab92-2f789f917316 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:02.148196 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-35407887-5ab4-4e0e-aad5-7450f0077877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:02.153103 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221de14-dd71-4e30-9fb1-0b212e6e19f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:02.153846 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221de14-dd71-4e30-9fb1-0b212e6e19f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:02.154131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221de14-dd71-4e30-9fb1-0b212e6e19f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:02.154557 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221de14-dd71-4e30-9fb1-0b212e6e19f8 admin admin] PATH_INFO: `/v3/users/c45f298350ba47e99cfa15e77f8cd702` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:02.155770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a221de14-dd71-4e30-9fb1-0b212e6e19f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c45f298350ba47e99cfa15e77f8cd702)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:02.157899 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a221de14-dd71-4e30-9fb1-0b212e6e19f8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:02.160592 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1002/2025] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:02.160592 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35407887-5ab4-4e0e-aad5-7450f0077877 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:02.176475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35407887-5ab4-4e0e-aad5-7450f0077877 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:02.181751 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5eaf3-ba20-49df-80d2-0c059491159b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:02.183193 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5eaf3-ba20-49df-80d2-0c059491159b admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:02.183902 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5eaf3-ba20-49df-80d2-0c059491159b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:02.191915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5eaf3-ba20-49df-80d2-0c059491159b admin admin] PATH_INFO: `/v3/projects/53df71b1486e462291225fa8f39ea8f5/users/89bcf65a87d94af593c0e6b7b22432e7/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:02.201873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5eaf3-ba20-49df-80d2-0c059491159b admin admin] RBAC: Authorizing `identity:create_grant(project_id=53df71b1486e462291225fa8f39ea8f5, user_id=89bcf65a87d94af593c0e6b7b22432e7, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:02.204563 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5eaf3-ba20-49df-80d2-0c059491159b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:02.299867 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1003/2026] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:43:02 2022] PUT /identity/v3/projects/53df71b1486e462291225fa8f39ea8f5/users/89bcf65a87d94af593c0e6b7b22432e7/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:02.300705 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5eaf3-ba20-49df-80d2-0c059491159b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:02.359231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a221de14-dd71-4e30-9fb1-0b212e6e19f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c45f298350ba47e99cfa15e77f8cd702'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:02.361556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5eaf3-ba20-49df-80d2-0c059491159b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:02.366599 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007b0ea-546b-4a60-b8e8-11b5ffc47b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:02.367234 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007b0ea-546b-4a60-b8e8-11b5ffc47b01 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:02.367376 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a221de14-dd71-4e30-9fb1-0b212e6e19f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c45f298350ba47e99cfa15e77f8cd702'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:02.367517 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007b0ea-546b-4a60-b8e8-11b5ffc47b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:02.367788 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007b0ea-546b-4a60-b8e8-11b5ffc47b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:02.368421 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007b0ea-546b-4a60-b8e8-11b5ffc47b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:02.370689 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007b0ea-546b-4a60-b8e8-11b5ffc47b01 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:02.380579 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1004/2027] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:43:02.381355 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007b0ea-546b-4a60-b8e8-11b5ffc47b01 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:02.382975 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c007b0ea-546b-4a60-b8e8-11b5ffc47b01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:02.396240 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:02.439623 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007b0ea-546b-4a60-b8e8-11b5ffc47b01 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:02.443373 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2aa48e-1a2b-4450-a035-eb84162de482 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:02.443982 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2aa48e-1a2b-4450-a035-eb84162de482 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:02.444263 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2aa48e-1a2b-4450-a035-eb84162de482 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:02.444538 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2aa48e-1a2b-4450-a035-eb84162de482 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:02.453841 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2aa48e-1a2b-4450-a035-eb84162de482 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:02.453981 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a221de14-dd71-4e30-9fb1-0b212e6e19f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c45f298350ba47e99cfa15e77f8cd702'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:02.454543 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a221de14-dd71-4e30-9fb1-0b212e6e19f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c45f298350ba47e99cfa15e77f8cd702'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:02.455212 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a221de14-dd71-4e30-9fb1-0b212e6e19f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c45f298350ba47e99cfa15e77f8cd702'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:02.455319 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2aa48e-1a2b-4450-a035-eb84162de482 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:02.465330 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1005/2028] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:43:02.466518 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf7c27-9329-4786-b80a-06f95e506e2d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:02.466802 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf7c27-9329-4786-b80a-06f95e506e2d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:02.467079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf7c27-9329-4786-b80a-06f95e506e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:02.483224 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1024/2029] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:43:02 2022] DELETE /identity/v3/users/c45f298350ba47e99cfa15e77f8cd702 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:02.483904 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221de14-dd71-4e30-9fb1-0b212e6e19f8 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:02.485245 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a221de14-dd71-4e30-9fb1-0b212e6e19f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:02.492284 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=82,keystone:INSERT=8,keystone:DELETE=10 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:02.500341 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-74cf7c27-9329-4786-b80a-06f95e506e2d None None] MFA Rules not processed for user `89bcf65a87d94af593c0e6b7b22432e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:02.501585 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-74cf7c27-9329-4786-b80a-06f95e506e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:02.556929 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221de14-dd71-4e30-9fb1-0b212e6e19f8 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:02.561943 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa030a-a347-4f2d-9f38-12f7bcd5f6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:02.562611 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa030a-a347-4f2d-9f38-12f7bcd5f6ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:02.562890 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa030a-a347-4f2d-9f38-12f7bcd5f6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:02.563206 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa030a-a347-4f2d-9f38-12f7bcd5f6ba admin admin] PATH_INFO: `/v3/projects/c884ad8cd9634a509b9fad3ae45f7a8e/users/5208f88baea343c389855eb26df89b16/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:02.569219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1006/2030] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:02 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:02.571328 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ce47a-e1fe-4c14-a91d-1a65272abed6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:02.571328 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ce47a-e1fe-4c14-a91d-1a65272abed6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:02.571328 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ce47a-e1fe-4c14-a91d-1a65272abed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:02.578536 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fa030a-a347-4f2d-9f38-12f7bcd5f6ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=c884ad8cd9634a509b9fad3ae45f7a8e, user_id=5208f88baea343c389855eb26df89b16, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:02.581283 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fa030a-a347-4f2d-9f38-12f7bcd5f6ba admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:02.590775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-091ce47a-e1fe-4c14-a91d-1a65272abed6 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:02.592012 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-091ce47a-e1fe-4c14-a91d-1a65272abed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:02.599695 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1007/2031] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:02.602898 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ce47a-e1fe-4c14-a91d-1a65272abed6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:02.602898 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-091ce47a-e1fe-4c14-a91d-1a65272abed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:02.614613 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ce47a-e1fe-4c14-a91d-1a65272abed6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:02.619860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba832c-7e51-4400-8323-f1854542c62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:02.620503 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba832c-7e51-4400-8323-f1854542c62e admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:02.620782 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba832c-7e51-4400-8323-f1854542c62e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:02.621340 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba832c-7e51-4400-8323-f1854542c62e admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:02.621677 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba832c-7e51-4400-8323-f1854542c62e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:02.623828 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba832c-7e51-4400-8323-f1854542c62e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:02.654840 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1025/2032] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:43:02 2022] PUT /identity/v3/projects/c884ad8cd9634a509b9fad3ae45f7a8e/users/5208f88baea343c389855eb26df89b16/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:02.656341 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa030a-a347-4f2d-9f38-12f7bcd5f6ba admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:02.659127 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f2fa030a-a347-4f2d-9f38-12f7bcd5f6ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:02.714319 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa030a-a347-4f2d-9f38-12f7bcd5f6ba admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:02.718516 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa7697-5374-488f-8fe6-819ff5010f09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:02.719117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa7697-5374-488f-8fe6-819ff5010f09 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:02.719433 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa7697-5374-488f-8fe6-819ff5010f09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:02.719710 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa7697-5374-488f-8fe6-819ff5010f09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:02.753795 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1008/2033] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:43:02 2022] POST /identity/v3/projects => generated 401 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:43:02.754836 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba832c-7e51-4400-8323-f1854542c62e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:02.785324 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa7697-5374-488f-8fe6-819ff5010f09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:02.786759 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa7697-5374-488f-8fe6-819ff5010f09 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:02.799038 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1026/2034] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:43:02.800215 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa7697-5374-488f-8fe6-819ff5010f09 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:02.810789 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa7697-5374-488f-8fe6-819ff5010f09 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:02.837931 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba832c-7e51-4400-8323-f1854542c62e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:02.847831 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c3397-b1b5-441a-94f6-f88f196ff8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:02.848536 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c3397-b1b5-441a-94f6-f88f196ff8e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:02.850900 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c3397-b1b5-441a-94f6-f88f196ff8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:02.851683 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c3397-b1b5-441a-94f6-f88f196ff8e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:02.852041 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c3397-b1b5-441a-94f6-f88f196ff8e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:02.853719 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c3397-b1b5-441a-94f6-f88f196ff8e3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:02.877464 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe888d-ade1-499a-8422-ac5720442e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:02.878094 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe888d-ade1-499a-8422-ac5720442e77 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:02.878379 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe888d-ade1-499a-8422-ac5720442e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:02.878659 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe888d-ade1-499a-8422-ac5720442e77 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:02.879378 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbe888d-ade1-499a-8422-ac5720442e77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:02.881478 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbe888d-ade1-499a-8422-ac5720442e77 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:02.891558 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1009/2035] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:43:02.892883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c3397-b1b5-441a-94f6-f88f196ff8e3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:02.904101 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c3397-b1b5-441a-94f6-f88f196ff8e3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:02.913346 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4213f79-4e87-4df8-a618-bfc2164045c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:02.914031 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4213f79-4e87-4df8-a618-bfc2164045c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:02.914318 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4213f79-4e87-4df8-a618-bfc2164045c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:02.915094 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4213f79-4e87-4df8-a618-bfc2164045c6 admin admin] PATH_INFO: `/v3/projects/c884ad8cd9634a509b9fad3ae45f7a8e/users/5208f88baea343c389855eb26df89b16/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:02.918524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4213f79-4e87-4df8-a618-bfc2164045c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c884ad8cd9634a509b9fad3ae45f7a8e, user_id=5208f88baea343c389855eb26df89b16, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:02.921824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4213f79-4e87-4df8-a618-bfc2164045c6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:03.012286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1010/2036] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:43:02 2022] PUT /identity/v3/projects/c884ad8cd9634a509b9fad3ae45f7a8e/users/5208f88baea343c389855eb26df89b16/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:03.012286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef779bc-985b-4a98-87a8-271b4fa9eab9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:03.012286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef779bc-985b-4a98-87a8-271b4fa9eab9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:03.012286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef779bc-985b-4a98-87a8-271b4fa9eab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:03.012286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5ef779bc-985b-4a98-87a8-271b4fa9eab9 None None] MFA Rules not processed for user `5208f88baea343c389855eb26df89b16`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:03.013264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5ef779bc-985b-4a98-87a8-271b4fa9eab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:03.016719 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:03.048684 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1027/2037] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:43:02 2022] POST /identity/v3/users => generated 307 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:43:03.052795 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe888d-ade1-499a-8422-ac5720442e77 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:03.062929 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe888d-ade1-499a-8422-ac5720442e77 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:03.077912 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:03.087973 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1011/2038] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:02 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:03.087973 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef779bc-985b-4a98-87a8-271b4fa9eab9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:03.089201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5ef779bc-985b-4a98-87a8-271b4fa9eab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:03.145608 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423beca-5198-45e3-ae29-676b7270f20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:03.146330 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423beca-5198-45e3-ae29-676b7270f20c admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:03.146896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423beca-5198-45e3-ae29-676b7270f20c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:03.147332 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423beca-5198-45e3-ae29-676b7270f20c admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:03.148204 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c423beca-5198-45e3-ae29-676b7270f20c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:03.151254 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c423beca-5198-45e3-ae29-676b7270f20c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:03.165145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1028/2039] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:43:03.170280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423beca-5198-45e3-ae29-676b7270f20c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:03.172198 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c423beca-5198-45e3-ae29-676b7270f20c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:03.174322 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef779bc-985b-4a98-87a8-271b4fa9eab9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:03.179919 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a5f56b-e701-43e6-b0f5-443b00e71124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:03.181494 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5f56b-e701-43e6-b0f5-443b00e71124 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:03.181780 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5f56b-e701-43e6-b0f5-443b00e71124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:03.182059 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5f56b-e701-43e6-b0f5-443b00e71124 admin admin] PATH_INFO: `/v3/projects/31b578119d744d0cb945cf93479e74c5` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:03.184311 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a5f56b-e701-43e6-b0f5-443b00e71124 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31b578119d744d0cb945cf93479e74c5)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:03.186388 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a5f56b-e701-43e6-b0f5-443b00e71124 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:03.239058 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423beca-5198-45e3-ae29-676b7270f20c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:03.242887 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c5c34-c83c-4e4a-bb4f-bff7cf2a7911 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:03.243522 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c5c34-c83c-4e4a-bb4f-bff7cf2a7911 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:03.243801 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c5c34-c83c-4e4a-bb4f-bff7cf2a7911 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:03.244074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c5c34-c83c-4e4a-bb4f-bff7cf2a7911 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:03.253725 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c5c34-c83c-4e4a-bb4f-bff7cf2a7911 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:03.255576 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c5c34-c83c-4e4a-bb4f-bff7cf2a7911 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:03.266628 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1029/2040] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:03 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:43:03.267464 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c5c34-c83c-4e4a-bb4f-bff7cf2a7911 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:03.279155 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c5c34-c83c-4e4a-bb4f-bff7cf2a7911 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:03.293255 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-27a5f56b-e701-43e6-b0f5-443b00e71124 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31b578119d744d0cb945cf93479e74c5', 'request_id': 'req-27a5f56b-e701-43e6-b0f5-443b00e71124'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:03.299766 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-27a5f56b-e701-43e6-b0f5-443b00e71124 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31b578119d744d0cb945cf93479e74c5', 'request_id': 'req-27a5f56b-e701-43e6-b0f5-443b00e71124'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:03.336101 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e9c77-553c-4f83-9216-bd2ee5781f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:03.336750 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e9c77-553c-4f83-9216-bd2ee5781f82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:03.337028 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e9c77-553c-4f83-9216-bd2ee5781f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:03.337301 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e9c77-553c-4f83-9216-bd2ee5781f82 admin admin] PATH_INFO: `/v3/projects/266cf62ea24445608fd4914eb86e381a/users/011a7112c6624136b30928a1b52a63b8/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:03.353397 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e9c77-553c-4f83-9216-bd2ee5781f82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=266cf62ea24445608fd4914eb86e381a, user_id=011a7112c6624136b30928a1b52a63b8, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:03.360490 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e9c77-553c-4f83-9216-bd2ee5781f82 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:03.381343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-27a5f56b-e701-43e6-b0f5-443b00e71124 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31b578119d744d0cb945cf93479e74c5', 'request_id': 'req-27a5f56b-e701-43e6-b0f5-443b00e71124'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:03.396236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-27a5f56b-e701-43e6-b0f5-443b00e71124 admin admin] The token cache is being invalidate because project 31b578119d744d0cb945cf93479e74c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:43:03.396510 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-27a5f56b-e701-43e6-b0f5-443b00e71124 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:03.397197 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-27a5f56b-e701-43e6-b0f5-443b00e71124 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:03.398708 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1012/2041] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:43:03 2022] DELETE /identity/v3/projects/31b578119d744d0cb945cf93479e74c5 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:03.399613 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a5f56b-e701-43e6-b0f5-443b00e71124 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:03.401577 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-27a5f56b-e701-43e6-b0f5-443b00e71124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:03.414588 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a5f56b-e701-43e6-b0f5-443b00e71124 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:03.420603 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71813d43-7d8a-42ca-a09e-a9091a1df010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:03.421241 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71813d43-7d8a-42ca-a09e-a9091a1df010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:03.421579 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71813d43-7d8a-42ca-a09e-a9091a1df010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:03.421773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71813d43-7d8a-42ca-a09e-a9091a1df010 admin admin] PATH_INFO: `/v3/projects/b35b533969bf4d6b885ed34bf6d9a779` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:03.422913 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71813d43-7d8a-42ca-a09e-a9091a1df010 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b35b533969bf4d6b885ed34bf6d9a779)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:03.424886 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71813d43-7d8a-42ca-a09e-a9091a1df010 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:03.442854 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1030/2042] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:43:03 2022] PUT /identity/v3/projects/266cf62ea24445608fd4914eb86e381a/users/011a7112c6624136b30928a1b52a63b8/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:03.446736 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361f35c-70f4-41c3-a429-9228e8391b13 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:03.446736 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361f35c-70f4-41c3-a429-9228e8391b13 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:03.446736 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361f35c-70f4-41c3-a429-9228e8391b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:03.459758 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:03.466066 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-8361f35c-70f4-41c3-a429-9228e8391b13 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:03.469606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8361f35c-70f4-41c3-a429-9228e8391b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:03.525883 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1031/2043] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:03.527571 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361f35c-70f4-41c3-a429-9228e8391b13 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:03.528973 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8361f35c-70f4-41c3-a429-9228e8391b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:03.538947 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:03.548069 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361f35c-70f4-41c3-a429-9228e8391b13 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:03.554513 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-71813d43-7d8a-42ca-a09e-a9091a1df010 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b35b533969bf4d6b885ed34bf6d9a779', 'request_id': 'req-71813d43-7d8a-42ca-a09e-a9091a1df010'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:03.560903 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-71813d43-7d8a-42ca-a09e-a9091a1df010 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b35b533969bf4d6b885ed34bf6d9a779', 'request_id': 'req-71813d43-7d8a-42ca-a09e-a9091a1df010'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:03.601166 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1cb7f-a417-486c-ab45-8dba718ae46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:03.601773 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1cb7f-a417-486c-ab45-8dba718ae46a admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:03.602050 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1cb7f-a417-486c-ab45-8dba718ae46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:03.602323 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1cb7f-a417-486c-ab45-8dba718ae46a admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:03.602949 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1cb7f-a417-486c-ab45-8dba718ae46a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:03.604739 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1cb7f-a417-486c-ab45-8dba718ae46a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:03.614817 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1032/2044] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:43:03.616014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e0c91-9ef9-4dda-b823-74c5feead18a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:03.616294 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e0c91-9ef9-4dda-b823-74c5feead18a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:03.616561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e0c91-9ef9-4dda-b823-74c5feead18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:03.634928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d92e0c91-9ef9-4dda-b823-74c5feead18a None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:03.636943 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d92e0c91-9ef9-4dda-b823-74c5feead18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:03.644777 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1033/2045] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:03.648928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e0c91-9ef9-4dda-b823-74c5feead18a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:03.655528 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e0c91-9ef9-4dda-b823-74c5feead18a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:03.659291 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-71813d43-7d8a-42ca-a09e-a9091a1df010 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b35b533969bf4d6b885ed34bf6d9a779', 'request_id': 'req-71813d43-7d8a-42ca-a09e-a9091a1df010'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:03.660529 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e87916-8262-4f02-a915-9a63f85277c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:03.661179 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e87916-8262-4f02-a915-9a63f85277c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:03.661485 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e87916-8262-4f02-a915-9a63f85277c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:03.661747 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e87916-8262-4f02-a915-9a63f85277c2 admin admin] PATH_INFO: `/v3/projects/266cf62ea24445608fd4914eb86e381a/users/011a7112c6624136b30928a1b52a63b8/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:03.663828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e87916-8262-4f02-a915-9a63f85277c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=266cf62ea24445608fd4914eb86e381a, user_id=011a7112c6624136b30928a1b52a63b8, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:03.666466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e87916-8262-4f02-a915-9a63f85277c2 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:03.671783 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-71813d43-7d8a-42ca-a09e-a9091a1df010 admin admin] The token cache is being invalidate because project b35b533969bf4d6b885ed34bf6d9a779 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:43:03.672109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-71813d43-7d8a-42ca-a09e-a9091a1df010 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:03.672726 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-71813d43-7d8a-42ca-a09e-a9091a1df010 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:03.674456 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1013/2046] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:43:03 2022] DELETE /identity/v3/projects/b35b533969bf4d6b885ed34bf6d9a779 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:03.674995 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71813d43-7d8a-42ca-a09e-a9091a1df010 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:03.680578 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-71813d43-7d8a-42ca-a09e-a9091a1df010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:03.690685 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1034/2047] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:43:03 2022] PUT /identity/v3/projects/266cf62ea24445608fd4914eb86e381a/users/011a7112c6624136b30928a1b52a63b8/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:03.690685 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e87916-8262-4f02-a915-9a63f85277c2 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:03.690685 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-54e87916-8262-4f02-a915-9a63f85277c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:03.692073 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71813d43-7d8a-42ca-a09e-a9091a1df010 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:03.753261 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f318023-3e27-4922-ab22-5c207602dcc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:03.753912 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f318023-3e27-4922-ab22-5c207602dcc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:03.754197 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f318023-3e27-4922-ab22-5c207602dcc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:03.754475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f318023-3e27-4922-ab22-5c207602dcc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:03.754667 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e87916-8262-4f02-a915-9a63f85277c2 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:03.755126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f318023-3e27-4922-ab22-5c207602dcc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:43:03.755462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f318023-3e27-4922-ab22-5c207602dcc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:03.757158 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f318023-3e27-4922-ab22-5c207602dcc5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:03.759443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab430215-74f8-4687-add8-e3c358e631e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:03.760067 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab430215-74f8-4687-add8-e3c358e631e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:03.760360 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab430215-74f8-4687-add8-e3c358e631e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:03.761221 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab430215-74f8-4687-add8-e3c358e631e5 admin admin] PATH_INFO: `/v3/users/d89ab7e011284e179f9a202b54056999` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:03.762380 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab430215-74f8-4687-add8-e3c358e631e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d89ab7e011284e179f9a202b54056999)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:03.764331 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab430215-74f8-4687-add8-e3c358e631e5 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:03.769623 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1014/2048] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:43:03.771387 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56af047-e1cf-4d43-810a-261301234b91 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:03.771678 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56af047-e1cf-4d43-810a-261301234b91 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:03.772093 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56af047-e1cf-4d43-810a-261301234b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:03.801714 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b56af047-e1cf-4d43-810a-261301234b91 None None] MFA Rules not processed for user `011a7112c6624136b30928a1b52a63b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:03.808052 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b56af047-e1cf-4d43-810a-261301234b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:03.845630 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:03.873977 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1015/2049] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:03 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:03.874938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56af047-e1cf-4d43-810a-261301234b91 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:03.885462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56af047-e1cf-4d43-810a-261301234b91 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:03.890454 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44795dc7-793a-481d-9692-30d95a708e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:03.891056 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44795dc7-793a-481d-9692-30d95a708e7c admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:03.891363 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44795dc7-793a-481d-9692-30d95a708e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:03.891644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44795dc7-793a-481d-9692-30d95a708e7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:03.892332 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44795dc7-793a-481d-9692-30d95a708e7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:03.894327 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44795dc7-793a-481d-9692-30d95a708e7c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:03.949873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:04.024992 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ab430215-74f8-4687-add8-e3c358e631e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd89ab7e011284e179f9a202b54056999'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:04.033261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ab430215-74f8-4687-add8-e3c358e631e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd89ab7e011284e179f9a202b54056999'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:04.044908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1016/2050] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:43:03 2022] POST /identity/v3/projects => generated 363 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:43:04.044908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44795dc7-793a-481d-9692-30d95a708e7c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:04.049799 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-44795dc7-793a-481d-9692-30d95a708e7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:04.116547 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44795dc7-793a-481d-9692-30d95a708e7c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:04.120315 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ec9bc-be6b-4617-8919-cf9b35c2c146 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:04.120886 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ec9bc-be6b-4617-8919-cf9b35c2c146 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:04.121169 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ec9bc-be6b-4617-8919-cf9b35c2c146 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:04.121431 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ec9bc-be6b-4617-8919-cf9b35c2c146 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:04.143973 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ab430215-74f8-4687-add8-e3c358e631e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd89ab7e011284e179f9a202b54056999'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:04.144653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ab430215-74f8-4687-add8-e3c358e631e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd89ab7e011284e179f9a202b54056999'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:04.145291 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ab430215-74f8-4687-add8-e3c358e631e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd89ab7e011284e179f9a202b54056999'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:04.166848 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1035/2051] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:43:03 2022] DELETE /identity/v3/users/d89ab7e011284e179f9a202b54056999 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:04.167771 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab430215-74f8-4687-add8-e3c358e631e5 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:04.169245 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ab430215-74f8-4687-add8-e3c358e631e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:04.189234 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ec9bc-be6b-4617-8919-cf9b35c2c146 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:04.190434 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ec9bc-be6b-4617-8919-cf9b35c2c146 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:04.201887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1017/2052] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:04 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:43:04.240504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab430215-74f8-4687-add8-e3c358e631e5 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:04.247131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09efd1a-16ac-493c-b366-8b7a13b43960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:04.247792 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09efd1a-16ac-493c-b366-8b7a13b43960 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:04.248077 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09efd1a-16ac-493c-b366-8b7a13b43960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:04.248383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09efd1a-16ac-493c-b366-8b7a13b43960 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:04.249020 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09efd1a-16ac-493c-b366-8b7a13b43960 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:04.250963 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09efd1a-16ac-493c-b366-8b7a13b43960 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:04.380599 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ec9bc-be6b-4617-8919-cf9b35c2c146 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:04.382555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e49ec9bc-be6b-4617-8919-cf9b35c2c146 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:04.392501 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1036/2053] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:43:04 2022] POST /identity/v3/users => generated 282 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:43:04.394660 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09efd1a-16ac-493c-b366-8b7a13b43960 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:04.418238 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09efd1a-16ac-493c-b366-8b7a13b43960 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:04.435211 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276bb14-4d43-4394-8332-73834b25e27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:04.435833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276bb14-4d43-4394-8332-73834b25e27a admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:04.436120 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276bb14-4d43-4394-8332-73834b25e27a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:04.436400 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276bb14-4d43-4394-8332-73834b25e27a admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:04.437052 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8276bb14-4d43-4394-8332-73834b25e27a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:04.438667 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8276bb14-4d43-4394-8332-73834b25e27a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:04.473294 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1037/2054] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:43:04.473294 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276bb14-4d43-4394-8332-73834b25e27a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:04.479445 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8276bb14-4d43-4394-8332-73834b25e27a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:04.493215 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ec9bc-be6b-4617-8919-cf9b35c2c146 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:04.498401 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61563962-68e8-4428-ab89-d622a81f0481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:04.500033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61563962-68e8-4428-ab89-d622a81f0481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:04.500033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61563962-68e8-4428-ab89-d622a81f0481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:04.500033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61563962-68e8-4428-ab89-d622a81f0481 admin admin] PATH_INFO: `/v3/users/2770577a04484dfe965458658b9366c7` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:04.501065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61563962-68e8-4428-ab89-d622a81f0481 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2770577a04484dfe965458658b9366c7)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:04.503085 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61563962-68e8-4428-ab89-d622a81f0481 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:04.577033 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276bb14-4d43-4394-8332-73834b25e27a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:04.580483 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894c85c-4cd2-4053-adb2-7aa6799e774a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:04.581027 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894c85c-4cd2-4053-adb2-7aa6799e774a service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:04.581372 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894c85c-4cd2-4053-adb2-7aa6799e774a service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:04.581716 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894c85c-4cd2-4053-adb2-7aa6799e774a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:04.583987 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d894c85c-4cd2-4053-adb2-7aa6799e774a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:04.622531 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:04.667132 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d894c85c-4cd2-4053-adb2-7aa6799e774a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:04.668676 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d894c85c-4cd2-4053-adb2-7aa6799e774a service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:04.677936 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1038/2055] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:04 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:43:04.678753 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894c85c-4cd2-4053-adb2-7aa6799e774a service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:04.690339 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894c85c-4cd2-4053-adb2-7aa6799e774a service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:04.695727 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf972d-06c6-4cc3-a02c-11f8d35f52a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:04.696456 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf972d-06c6-4cc3-a02c-11f8d35f52a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:04.696612 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf972d-06c6-4cc3-a02c-11f8d35f52a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:04.696891 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf972d-06c6-4cc3-a02c-11f8d35f52a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:04.697592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf972d-06c6-4cc3-a02c-11f8d35f52a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:04.699981 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf972d-06c6-4cc3-a02c-11f8d35f52a7 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:04.714768 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1039/2056] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:43:04.714768 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf972d-06c6-4cc3-a02c-11f8d35f52a7 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:04.720123 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:INSERT=1,keystone:SELECT=92,keystone:DELETE=7 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:04.721961 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-61563962-68e8-4428-ab89-d622a81f0481 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2770577a04484dfe965458658b9366c7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:04.726920 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf972d-06c6-4cc3-a02c-11f8d35f52a7 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:04.732348 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768fed6-bbd1-40d1-a991-a93d3dbbc8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:04.733401 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768fed6-bbd1-40d1-a991-a93d3dbbc8cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:04.733877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768fed6-bbd1-40d1-a991-a93d3dbbc8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:04.734454 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768fed6-bbd1-40d1-a991-a93d3dbbc8cc admin admin] PATH_INFO: `/v3/projects/98abede88c5c4949ab7a373def1d2f7a/users/07056b0bd25047e189356dc54a29dc69/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:04.751809 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8768fed6-bbd1-40d1-a991-a93d3dbbc8cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=98abede88c5c4949ab7a373def1d2f7a, user_id=07056b0bd25047e189356dc54a29dc69, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:04.754433 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8768fed6-bbd1-40d1-a991-a93d3dbbc8cc admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:04.785374 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-61563962-68e8-4428-ab89-d622a81f0481 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2770577a04484dfe965458658b9366c7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:04.786048 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-61563962-68e8-4428-ab89-d622a81f0481 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2770577a04484dfe965458658b9366c7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:04.786704 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-61563962-68e8-4428-ab89-d622a81f0481 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2770577a04484dfe965458658b9366c7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:04.794829 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-61563962-68e8-4428-ab89-d622a81f0481 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2770577a04484dfe965458658b9366c7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:04.821297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1040/2057] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:43:04 2022] PUT /identity/v3/projects/98abede88c5c4949ab7a373def1d2f7a/users/07056b0bd25047e189356dc54a29dc69/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:43:04.830860 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768fed6-bbd1-40d1-a991-a93d3dbbc8cc admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:04.836090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8768fed6-bbd1-40d1-a991-a93d3dbbc8cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:04.844829 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1018/2058] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:43:04 2022] DELETE /identity/v3/users/2770577a04484dfe965458658b9366c7 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:04.854885 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978356be-6d98-47f5-9c78-9dc9e9f1bbfe None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:04.854885 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978356be-6d98-47f5-9c78-9dc9e9f1bbfe None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:04.854885 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978356be-6d98-47f5-9c78-9dc9e9f1bbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:04.882054 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-978356be-6d98-47f5-9c78-9dc9e9f1bbfe None None] MFA Rules not processed for user `07056b0bd25047e189356dc54a29dc69`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:04.885215 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-978356be-6d98-47f5-9c78-9dc9e9f1bbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:04.908754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768fed6-bbd1-40d1-a991-a93d3dbbc8cc admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:04.955913 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1019/2059] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:04.957319 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f4a2e-5b2f-4c41-82d3-92f8209c0c94 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:04.957700 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f4a2e-5b2f-4c41-82d3-92f8209c0c94 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:04.958689 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f4a2e-5b2f-4c41-82d3-92f8209c0c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:04.981206 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-496f4a2e-5b2f-4c41-82d3-92f8209c0c94 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:04.982691 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb126c-548d-4c6b-a8e2-c86692925359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:04.982870 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-496f4a2e-5b2f-4c41-82d3-92f8209c0c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:04.983369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb126c-548d-4c6b-a8e2-c86692925359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:04.983653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb126c-548d-4c6b-a8e2-c86692925359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:04.983928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb126c-548d-4c6b-a8e2-c86692925359 admin admin] PATH_INFO: `/v3/projects/4d7d1ca0a8db42429b4e4bbae2dd401a` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:04.985060 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb126c-548d-4c6b-a8e2-c86692925359 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d7d1ca0a8db42429b4e4bbae2dd401a)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:04.987262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb126c-548d-4c6b-a8e2-c86692925359 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:04.995658 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1020/2060] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:04.995658 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f4a2e-5b2f-4c41-82d3-92f8209c0c94 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:04.995658 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-496f4a2e-5b2f-4c41-82d3-92f8209c0c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:05.053548 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f4a2e-5b2f-4c41-82d3-92f8209c0c94 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:05.057274 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d9163-4c4a-4b14-a2c3-19d81d05540c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:05.058051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d9163-4c4a-4b14-a2c3-19d81d05540c service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:05.058337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d9163-4c4a-4b14-a2c3-19d81d05540c service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:05.058618 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d9163-4c4a-4b14-a2c3-19d81d05540c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:05.067304 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88d9163-4c4a-4b14-a2c3-19d81d05540c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:05.068771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88d9163-4c4a-4b14-a2c3-19d81d05540c service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:05.078352 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1021/2061] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:43:05.079000 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d9163-4c4a-4b14-a2c3-19d81d05540c service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:05.080381 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f88d9163-4c4a-4b14-a2c3-19d81d05540c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:05.133746 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bcfb126c-548d-4c6b-a8e2-c86692925359 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d7d1ca0a8db42429b4e4bbae2dd401a', 'request_id': 'req-bcfb126c-548d-4c6b-a8e2-c86692925359'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:05.141200 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bcfb126c-548d-4c6b-a8e2-c86692925359 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d7d1ca0a8db42429b4e4bbae2dd401a', 'request_id': 'req-bcfb126c-548d-4c6b-a8e2-c86692925359'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:05.162974 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d9163-4c4a-4b14-a2c3-19d81d05540c service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:05.219465 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4032f6-927a-4038-8ea3-593bd660c325 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:05.220088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4032f6-927a-4038-8ea3-593bd660c325 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:05.220371 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4032f6-927a-4038-8ea3-593bd660c325 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:05.220998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4032f6-927a-4038-8ea3-593bd660c325 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:05.231453 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bcfb126c-548d-4c6b-a8e2-c86692925359 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d7d1ca0a8db42429b4e4bbae2dd401a', 'request_id': 'req-bcfb126c-548d-4c6b-a8e2-c86692925359'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:05.247159 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bcfb126c-548d-4c6b-a8e2-c86692925359 admin admin] The token cache is being invalidate because project 4d7d1ca0a8db42429b4e4bbae2dd401a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:43:05.247583 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bcfb126c-548d-4c6b-a8e2-c86692925359 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:05.248337 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bcfb126c-548d-4c6b-a8e2-c86692925359 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:05.249913 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1041/2062] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:43:04 2022] DELETE /identity/v3/projects/4d7d1ca0a8db42429b4e4bbae2dd401a => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:05.253801 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb126c-548d-4c6b-a8e2-c86692925359 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:05.255563 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bcfb126c-548d-4c6b-a8e2-c86692925359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:05.287583 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4032f6-927a-4038-8ea3-593bd660c325 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:05.289029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4032f6-927a-4038-8ea3-593bd660c325 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:05.300583 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1022/2063] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:43:05.333627 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb126c-548d-4c6b-a8e2-c86692925359 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:05.339249 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8bafda-5234-4d6a-a2e4-2b543ff277a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:05.339884 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bafda-5234-4d6a-a2e4-2b543ff277a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:05.340162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bafda-5234-4d6a-a2e4-2b543ff277a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:05.340434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bafda-5234-4d6a-a2e4-2b543ff277a4 admin admin] PATH_INFO: `/v3/users/848bac99d20944dca78f86f666565bd1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:05.341577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8bafda-5234-4d6a-a2e4-2b543ff277a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=848bac99d20944dca78f86f666565bd1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:05.343543 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8bafda-5234-4d6a-a2e4-2b543ff277a4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:05.610496 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-eb8bafda-5234-4d6a-a2e4-2b543ff277a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '848bac99d20944dca78f86f666565bd1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:05.617827 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-eb8bafda-5234-4d6a-a2e4-2b543ff277a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '848bac99d20944dca78f86f666565bd1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:05.649798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:05.762802 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-eb8bafda-5234-4d6a-a2e4-2b543ff277a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '848bac99d20944dca78f86f666565bd1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:05.763692 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-eb8bafda-5234-4d6a-a2e4-2b543ff277a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '848bac99d20944dca78f86f666565bd1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:05.764318 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-eb8bafda-5234-4d6a-a2e4-2b543ff277a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '848bac99d20944dca78f86f666565bd1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:05.783099 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1042/2064] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:43:05 2022] DELETE /identity/v3/users/848bac99d20944dca78f86f666565bd1 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:43:05.816528 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4032f6-927a-4038-8ea3-593bd660c325 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:05.818089 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0b4032f6-927a-4038-8ea3-593bd660c325 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:05.845987 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4032f6-927a-4038-8ea3-593bd660c325 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:05.915866 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0829dd-d770-4e2b-b214-0b567149bd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:05.917173 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0829dd-d770-4e2b-b214-0b567149bd0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:05.917472 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0829dd-d770-4e2b-b214-0b567149bd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:05.918980 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:05.919498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0829dd-d770-4e2b-b214-0b567149bd0f admin admin] PATH_INFO: `/v3/projects/639a245c5a734323bd99f6eb2738a844` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:05.921079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0829dd-d770-4e2b-b214-0b567149bd0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=639a245c5a734323bd99f6eb2738a844)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:05.923016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0829dd-d770-4e2b-b214-0b567149bd0f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:06.060232 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2d0829dd-d770-4e2b-b214-0b567149bd0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '639a245c5a734323bd99f6eb2738a844', 'request_id': 'req-2d0829dd-d770-4e2b-b214-0b567149bd0f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:06.066529 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2d0829dd-d770-4e2b-b214-0b567149bd0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '639a245c5a734323bd99f6eb2738a844', 'request_id': 'req-2d0829dd-d770-4e2b-b214-0b567149bd0f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:06.132813 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2d0829dd-d770-4e2b-b214-0b567149bd0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '639a245c5a734323bd99f6eb2738a844', 'request_id': 'req-2d0829dd-d770-4e2b-b214-0b567149bd0f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:06.146746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2d0829dd-d770-4e2b-b214-0b567149bd0f admin admin] The token cache is being invalidate because project 639a245c5a734323bd99f6eb2738a844 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:43:06.147134 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2d0829dd-d770-4e2b-b214-0b567149bd0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:06.148282 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2d0829dd-d770-4e2b-b214-0b567149bd0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:06.149748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1023/2065] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:43:05 2022] DELETE /identity/v3/projects/639a245c5a734323bd99f6eb2738a844 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:06.153560 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8bafda-5234-4d6a-a2e4-2b543ff277a4 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:06.155147 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-eb8bafda-5234-4d6a-a2e4-2b543ff277a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:06.169165 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8bafda-5234-4d6a-a2e4-2b543ff277a4 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:06.226211 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb4cf8-ab22-4382-ba6e-f33a0ece3fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:06.226838 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb4cf8-ab22-4382-ba6e-f33a0ece3fe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:06.227120 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb4cf8-ab22-4382-ba6e-f33a0ece3fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:06.227425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb4cf8-ab22-4382-ba6e-f33a0ece3fe9 admin admin] PATH_INFO: `/v3/users/5208f88baea343c389855eb26df89b16` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:06.228617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb4cf8-ab22-4382-ba6e-f33a0ece3fe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5208f88baea343c389855eb26df89b16)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:06.231092 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb4cf8-ab22-4382-ba6e-f33a0ece3fe9 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:06.249012 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0829dd-d770-4e2b-b214-0b567149bd0f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:06.250723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2d0829dd-d770-4e2b-b214-0b567149bd0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:06.278777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0829dd-d770-4e2b-b214-0b567149bd0f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:06.278777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a75380e-cb52-4118-a2b8-91be55f9f989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:06.278777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75380e-cb52-4118-a2b8-91be55f9f989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:06.278777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75380e-cb52-4118-a2b8-91be55f9f989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:06.279572 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75380e-cb52-4118-a2b8-91be55f9f989 admin admin] PATH_INFO: `/v3/projects/7d57d9ef2e5d4c80a22d8082d32c0dc2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:06.279572 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a75380e-cb52-4118-a2b8-91be55f9f989 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d57d9ef2e5d4c80a22d8082d32c0dc2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:06.279572 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a75380e-cb52-4118-a2b8-91be55f9f989 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:06.432770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-7fcb4cf8-ab22-4382-ba6e-f33a0ece3fe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5208f88baea343c389855eb26df89b16'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:06.433009 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5a75380e-cb52-4118-a2b8-91be55f9f989 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d57d9ef2e5d4c80a22d8082d32c0dc2', 'request_id': 'req-5a75380e-cb52-4118-a2b8-91be55f9f989'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:06.438661 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5a75380e-cb52-4118-a2b8-91be55f9f989 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d57d9ef2e5d4c80a22d8082d32c0dc2', 'request_id': 'req-5a75380e-cb52-4118-a2b8-91be55f9f989'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:06.440048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-7fcb4cf8-ab22-4382-ba6e-f33a0ece3fe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5208f88baea343c389855eb26df89b16'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:06.522442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5a75380e-cb52-4118-a2b8-91be55f9f989 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d57d9ef2e5d4c80a22d8082d32c0dc2', 'request_id': 'req-5a75380e-cb52-4118-a2b8-91be55f9f989'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:06.535639 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5a75380e-cb52-4118-a2b8-91be55f9f989 admin admin] The token cache is being invalidate because project 7d57d9ef2e5d4c80a22d8082d32c0dc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:43:06.535933 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5a75380e-cb52-4118-a2b8-91be55f9f989 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:06.536603 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5a75380e-cb52-4118-a2b8-91be55f9f989 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:06.538305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1024/2066] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:43:06 2022] DELETE /identity/v3/projects/7d57d9ef2e5d4c80a22d8082d32c0dc2 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:06.619977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-7fcb4cf8-ab22-4382-ba6e-f33a0ece3fe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5208f88baea343c389855eb26df89b16'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:06.620922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-7fcb4cf8-ab22-4382-ba6e-f33a0ece3fe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5208f88baea343c389855eb26df89b16'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:06.621620 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-7fcb4cf8-ab22-4382-ba6e-f33a0ece3fe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5208f88baea343c389855eb26df89b16'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:06.646322 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1043/2067] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:43:06 2022] DELETE /identity/v3/users/5208f88baea343c389855eb26df89b16 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:07.133789 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:07.140873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a75380e-cb52-4118-a2b8-91be55f9f989 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:07.142390 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5a75380e-cb52-4118-a2b8-91be55f9f989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:07.211569 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a75380e-cb52-4118-a2b8-91be55f9f989 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:07.272162 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:07.279512 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a740551-5554-40bb-93b1-1276cd9c2641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:07.280816 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a740551-5554-40bb-93b1-1276cd9c2641 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:07.280816 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a740551-5554-40bb-93b1-1276cd9c2641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:07.280816 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a740551-5554-40bb-93b1-1276cd9c2641 admin admin] PATH_INFO: `/v3/projects/c884ad8cd9634a509b9fad3ae45f7a8e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:07.281987 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a740551-5554-40bb-93b1-1276cd9c2641 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c884ad8cd9634a509b9fad3ae45f7a8e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:07.284003 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a740551-5554-40bb-93b1-1276cd9c2641 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:07.417152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7a740551-5554-40bb-93b1-1276cd9c2641 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c884ad8cd9634a509b9fad3ae45f7a8e', 'request_id': 'req-7a740551-5554-40bb-93b1-1276cd9c2641'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:07.422815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7a740551-5554-40bb-93b1-1276cd9c2641 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c884ad8cd9634a509b9fad3ae45f7a8e', 'request_id': 'req-7a740551-5554-40bb-93b1-1276cd9c2641'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:07.486660 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7a740551-5554-40bb-93b1-1276cd9c2641 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c884ad8cd9634a509b9fad3ae45f7a8e', 'request_id': 'req-7a740551-5554-40bb-93b1-1276cd9c2641'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:07.498933 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7a740551-5554-40bb-93b1-1276cd9c2641 admin admin] The token cache is being invalidate because project c884ad8cd9634a509b9fad3ae45f7a8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:43:07.499434 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7a740551-5554-40bb-93b1-1276cd9c2641 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:07.500081 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7a740551-5554-40bb-93b1-1276cd9c2641 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:07.501604 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1025/2068] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:43:07 2022] DELETE /identity/v3/projects/c884ad8cd9634a509b9fad3ae45f7a8e => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:43:07.506631 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a67d8-da42-43de-8346-2c5d5d6f9cb4 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:07.507095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a67d8-da42-43de-8346-2c5d5d6f9cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:07.507439 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a67d8-da42-43de-8346-2c5d5d6f9cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:07.528775 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-745a67d8-da42-43de-8346-2c5d5d6f9cb4 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:07.529981 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-745a67d8-da42-43de-8346-2c5d5d6f9cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:07.587424 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1044/2069] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:43:07.594970 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166f6fb-7f81-4a63-a37d-4f1889052553 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:07.595299 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166f6fb-7f81-4a63-a37d-4f1889052553 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:07.595580 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166f6fb-7f81-4a63-a37d-4f1889052553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:07.615725 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9166f6fb-7f81-4a63-a37d-4f1889052553 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:07.616939 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9166f6fb-7f81-4a63-a37d-4f1889052553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:07.627401 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1026/2070] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:07.631815 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a67d8-da42-43de-8346-2c5d5d6f9cb4 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:07.642872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a67d8-da42-43de-8346-2c5d5d6f9cb4 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:07.647563 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1926d08-2c10-4e49-9c5a-d9d08e757a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:07.648180 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1926d08-2c10-4e49-9c5a-d9d08e757a3a admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:07.650537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1926d08-2c10-4e49-9c5a-d9d08e757a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:07.650537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1926d08-2c10-4e49-9c5a-d9d08e757a3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:07.650537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1926d08-2c10-4e49-9c5a-d9d08e757a3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:43:07.650537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1926d08-2c10-4e49-9c5a-d9d08e757a3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:07.651283 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1926d08-2c10-4e49-9c5a-d9d08e757a3a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:07.662468 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1045/2071] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:43:07.667565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166f6fb-7f81-4a63-a37d-4f1889052553 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:07.677094 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166f6fb-7f81-4a63-a37d-4f1889052553 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:07.681866 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeb1f30-612d-4d02-b54c-fa4eed73af5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:07.687618 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb1f30-612d-4d02-b54c-fa4eed73af5a admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:07.687618 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb1f30-612d-4d02-b54c-fa4eed73af5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:07.687618 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb1f30-612d-4d02-b54c-fa4eed73af5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:07.687618 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeb1f30-612d-4d02-b54c-fa4eed73af5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:07.687618 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeb1f30-612d-4d02-b54c-fa4eed73af5a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:07.739843 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:07.763740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1027/2072] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:43:07 2022] POST /identity/v3/projects => generated 357 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:43:07.767928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1926d08-2c10-4e49-9c5a-d9d08e757a3a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:07.778807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1926d08-2c10-4e49-9c5a-d9d08e757a3a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:07.822048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:07.834417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c15b4-6b6b-4081-aba2-2eb10ed9bc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:07.835013 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c15b4-6b6b-4081-aba2-2eb10ed9bc19 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:07.835327 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c15b4-6b6b-4081-aba2-2eb10ed9bc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:07.835599 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c15b4-6b6b-4081-aba2-2eb10ed9bc19 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:07.836222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c15b4-6b6b-4081-aba2-2eb10ed9bc19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:07.838172 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c15b4-6b6b-4081-aba2-2eb10ed9bc19 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:07.940147 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1046/2073] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:43:07 2022] POST /identity/v3/users => generated 279 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:43:07.943859 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeb1f30-612d-4d02-b54c-fa4eed73af5a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:07.954978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeb1f30-612d-4d02-b54c-fa4eed73af5a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:07.960069 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab21cc-5d0c-47f8-9896-86826ff25273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:07.960803 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab21cc-5d0c-47f8-9896-86826ff25273 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:07.961134 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab21cc-5d0c-47f8-9896-86826ff25273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:07.961450 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab21cc-5d0c-47f8-9896-86826ff25273 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:07.962130 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab21cc-5d0c-47f8-9896-86826ff25273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:07.963790 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab21cc-5d0c-47f8-9896-86826ff25273 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:07.977929 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1028/2074] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:43:07.979341 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c15b4-6b6b-4081-aba2-2eb10ed9bc19 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:07.990486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c15b4-6b6b-4081-aba2-2eb10ed9bc19 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:07.995935 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78c0e0-b585-4e62-85f2-4554b941d134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:07.996181 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78c0e0-b585-4e62-85f2-4554b941d134 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:07.996462 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78c0e0-b585-4e62-85f2-4554b941d134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:07.997030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78c0e0-b585-4e62-85f2-4554b941d134 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:07.997679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78c0e0-b585-4e62-85f2-4554b941d134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:07.999311 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78c0e0-b585-4e62-85f2-4554b941d134 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:08.009963 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1047/2075] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:43:08.014759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab21cc-5d0c-47f8-9896-86826ff25273 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:08.024782 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab21cc-5d0c-47f8-9896-86826ff25273 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:08.030490 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509c485-fb9d-47f2-92a7-0ba756c65538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:08.031169 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509c485-fb9d-47f2-92a7-0ba756c65538 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:08.031519 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509c485-fb9d-47f2-92a7-0ba756c65538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:08.031798 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509c485-fb9d-47f2-92a7-0ba756c65538 admin admin] PATH_INFO: `/v3/projects/0d8eba4e4b1f4017b35f87797c09854d/users/0384636148ec4da5999e2cac62de2165/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:08.047214 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509c485-fb9d-47f2-92a7-0ba756c65538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d8eba4e4b1f4017b35f87797c09854d, user_id=0384636148ec4da5999e2cac62de2165, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:08.049864 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509c485-fb9d-47f2-92a7-0ba756c65538 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:08.128700 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1029/2076] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:43:08 2022] PUT /identity/v3/projects/0d8eba4e4b1f4017b35f87797c09854d/users/0384636148ec4da5999e2cac62de2165/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:08.140278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d21d0-1b27-4278-85a7-0b1ce368ecf4 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:08.140367 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d21d0-1b27-4278-85a7-0b1ce368ecf4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:08.141290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d21d0-1b27-4278-85a7-0b1ce368ecf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:08.180479 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3b3d21d0-1b27-4278-85a7-0b1ce368ecf4 None None] MFA Rules not processed for user `0384636148ec4da5999e2cac62de2165`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:08.181733 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3b3d21d0-1b27-4278-85a7-0b1ce368ecf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:08.259231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1048/2077] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:08 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:08.260596 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509c485-fb9d-47f2-92a7-0ba756c65538 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:08.329754 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509c485-fb9d-47f2-92a7-0ba756c65538 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:08.334496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6fc66c-6813-4ec9-a5ac-4525789ebc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:08.335246 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6fc66c-6813-4ec9-a5ac-4525789ebc44 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:08.335534 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6fc66c-6813-4ec9-a5ac-4525789ebc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:08.335804 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6fc66c-6813-4ec9-a5ac-4525789ebc44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:08.336467 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6fc66c-6813-4ec9-a5ac-4525789ebc44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:08.338363 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6fc66c-6813-4ec9-a5ac-4525789ebc44 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:08.443477 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1030/2078] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:43:08 2022] POST /identity/v3/projects => generated 357 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:43:08.447714 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d21d0-1b27-4278-85a7-0b1ce368ecf4 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:08.459822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d21d0-1b27-4278-85a7-0b1ce368ecf4 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:08.519891 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8991a4-1390-4c9c-bd53-68ab2d5144fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:08.520557 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8991a4-1390-4c9c-bd53-68ab2d5144fb admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:08.520776 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8991a4-1390-4c9c-bd53-68ab2d5144fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:08.521051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8991a4-1390-4c9c-bd53-68ab2d5144fb admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:08.521725 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8991a4-1390-4c9c-bd53-68ab2d5144fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:08.523642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8991a4-1390-4c9c-bd53-68ab2d5144fb admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:08.613995 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1049/2079] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:43:08 2022] POST /identity/v3/users => generated 285 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:43:08.618324 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6fc66c-6813-4ec9-a5ac-4525789ebc44 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:08.629440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6fc66c-6813-4ec9-a5ac-4525789ebc44 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:08.634373 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4c6e7-d149-4851-b49e-504c4385bee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:08.635086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4c6e7-d149-4851-b49e-504c4385bee7 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:08.635406 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4c6e7-d149-4851-b49e-504c4385bee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:08.635693 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4c6e7-d149-4851-b49e-504c4385bee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:08.636473 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4c6e7-d149-4851-b49e-504c4385bee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:08.638855 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4c6e7-d149-4851-b49e-504c4385bee7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:08.649675 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1031/2080] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:43:08.653914 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8991a4-1390-4c9c-bd53-68ab2d5144fb admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:08.664183 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8991a4-1390-4c9c-bd53-68ab2d5144fb admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:08.669030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f7009-851a-4c81-bee5-b3106387bd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:08.669713 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f7009-851a-4c81-bee5-b3106387bd72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:08.670010 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f7009-851a-4c81-bee5-b3106387bd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:08.670340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f7009-851a-4c81-bee5-b3106387bd72 admin admin] PATH_INFO: `/v3/projects/f36052743f104c17b530d20339d1f1e1/users/c3a6d87ff1224b048a9fea81dfbe2e50/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:08.686367 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105f7009-851a-4c81-bee5-b3106387bd72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f36052743f104c17b530d20339d1f1e1, user_id=c3a6d87ff1224b048a9fea81dfbe2e50, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:08.689084 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105f7009-851a-4c81-bee5-b3106387bd72 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:08.784837 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1050/2081] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:43:08 2022] PUT /identity/v3/projects/f36052743f104c17b530d20339d1f1e1/users/c3a6d87ff1224b048a9fea81dfbe2e50/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:08.788874 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4c6e7-d149-4851-b49e-504c4385bee7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:08.799201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4c6e7-d149-4851-b49e-504c4385bee7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:08.811150 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:08.854921 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25feb96d-bcb2-4fbe-a816-4b78abc3dae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:08.856052 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25feb96d-bcb2-4fbe-a816-4b78abc3dae0 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:08.856521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25feb96d-bcb2-4fbe-a816-4b78abc3dae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:08.856873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25feb96d-bcb2-4fbe-a816-4b78abc3dae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:08.857507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25feb96d-bcb2-4fbe-a816-4b78abc3dae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:08.859128 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25feb96d-bcb2-4fbe-a816-4b78abc3dae0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:08.871805 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1032/2082] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:43:08.873919 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f7009-851a-4c81-bee5-b3106387bd72 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:08.884884 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f7009-851a-4c81-bee5-b3106387bd72 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:08.889601 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f16b5-b02e-4c34-ab0d-ea234398a742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:08.890300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f16b5-b02e-4c34-ab0d-ea234398a742 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:08.890582 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f16b5-b02e-4c34-ab0d-ea234398a742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:08.890850 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f16b5-b02e-4c34-ab0d-ea234398a742 admin admin] PATH_INFO: `/v3/projects/f36052743f104c17b530d20339d1f1e1/users/c3a6d87ff1224b048a9fea81dfbe2e50/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:08.892877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49f16b5-b02e-4c34-ab0d-ea234398a742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f36052743f104c17b530d20339d1f1e1, user_id=c3a6d87ff1224b048a9fea81dfbe2e50, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:08.895602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49f16b5-b02e-4c34-ab0d-ea234398a742 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:08.905877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1051/2083] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:43:08 2022] PUT /identity/v3/projects/f36052743f104c17b530d20339d1f1e1/users/c3a6d87ff1224b048a9fea81dfbe2e50/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:08.917028 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af75162-a0d8-426d-97a2-b5b255fd686e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:08.917315 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af75162-a0d8-426d-97a2-b5b255fd686e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:08.917590 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af75162-a0d8-426d-97a2-b5b255fd686e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:08.946693 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4af75162-a0d8-426d-97a2-b5b255fd686e None None] MFA Rules not processed for user `c3a6d87ff1224b048a9fea81dfbe2e50`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:08.947921 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4af75162-a0d8-426d-97a2-b5b255fd686e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:09.021497 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1033/2084] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:08 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:09.034172 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f16b5-b02e-4c34-ab0d-ea234398a742 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:09.036052 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b49f16b5-b02e-4c34-ab0d-ea234398a742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:09.047134 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:09.095982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f16b5-b02e-4c34-ab0d-ea234398a742 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:09.099872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f8647-e77b-4c11-993b-8c25164c70b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:09.100513 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f8647-e77b-4c11-993b-8c25164c70b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:09.100820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f8647-e77b-4c11-993b-8c25164c70b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:09.101090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f8647-e77b-4c11-993b-8c25164c70b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:09.110721 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f8647-e77b-4c11-993b-8c25164c70b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:09.113020 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f8647-e77b-4c11-993b-8c25164c70b8 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:09.122899 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1052/2085] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:09 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:43:09.324556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af75162-a0d8-426d-97a2-b5b255fd686e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:09.334651 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af75162-a0d8-426d-97a2-b5b255fd686e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:09.338497 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb444220-13d3-4cbe-9da2-6b7100c9004b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:09.339204 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb444220-13d3-4cbe-9da2-6b7100c9004b service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:09.339569 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb444220-13d3-4cbe-9da2-6b7100c9004b service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:09.340645 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb444220-13d3-4cbe-9da2-6b7100c9004b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:09.404258 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb444220-13d3-4cbe-9da2-6b7100c9004b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:09.406110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb444220-13d3-4cbe-9da2-6b7100c9004b service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:09.415870 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1034/2086] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:09 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:43:09.876998 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f8647-e77b-4c11-993b-8c25164c70b8 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:09.878451 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-dc7f8647-e77b-4c11-993b-8c25164c70b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:09.942633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f8647-e77b-4c11-993b-8c25164c70b8 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:09.947224 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3bad51-ef55-4ef1-8c8e-de0a231d636c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:09.947860 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3bad51-ef55-4ef1-8c8e-de0a231d636c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:09.949639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3bad51-ef55-4ef1-8c8e-de0a231d636c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:09.949639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3bad51-ef55-4ef1-8c8e-de0a231d636c admin admin] PATH_INFO: `/v3/users/89bcf65a87d94af593c0e6b7b22432e7` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:09.949639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3bad51-ef55-4ef1-8c8e-de0a231d636c admin admin] RBAC: Authorizing `identity:delete_user(user_id=89bcf65a87d94af593c0e6b7b22432e7)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:09.951502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3bad51-ef55-4ef1-8c8e-de0a231d636c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:10.140129 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-af3bad51-ef55-4ef1-8c8e-de0a231d636c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89bcf65a87d94af593c0e6b7b22432e7'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:10.146883 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-af3bad51-ef55-4ef1-8c8e-de0a231d636c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89bcf65a87d94af593c0e6b7b22432e7'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:10.212351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-af3bad51-ef55-4ef1-8c8e-de0a231d636c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89bcf65a87d94af593c0e6b7b22432e7'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:10.212977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-af3bad51-ef55-4ef1-8c8e-de0a231d636c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89bcf65a87d94af593c0e6b7b22432e7'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:10.213591 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-af3bad51-ef55-4ef1-8c8e-de0a231d636c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89bcf65a87d94af593c0e6b7b22432e7'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:10.233089 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1053/2087] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:43:09 2022] DELETE /identity/v3/users/89bcf65a87d94af593c0e6b7b22432e7 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:10.238109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123a5ed-5403-4307-84bf-bcc28a0607b9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:10.238394 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123a5ed-5403-4307-84bf-bcc28a0607b9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:10.238696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123a5ed-5403-4307-84bf-bcc28a0607b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:10.258232 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4123a5ed-5403-4307-84bf-bcc28a0607b9 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:10.259504 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4123a5ed-5403-4307-84bf-bcc28a0607b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:10.290307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:10.318379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1035/2088] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:10.328788 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3bad51-ef55-4ef1-8c8e-de0a231d636c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:10.330191 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-af3bad51-ef55-4ef1-8c8e-de0a231d636c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:10.361995 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:10.385627 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3bad51-ef55-4ef1-8c8e-de0a231d636c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:10.390031 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05781ba5-4142-496c-b091-c3919edb9e98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:10.390675 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05781ba5-4142-496c-b091-c3919edb9e98 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:10.390904 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05781ba5-4142-496c-b091-c3919edb9e98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:10.391201 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05781ba5-4142-496c-b091-c3919edb9e98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:10.399980 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05781ba5-4142-496c-b091-c3919edb9e98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:10.401427 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05781ba5-4142-496c-b091-c3919edb9e98 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:10.413082 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1054/2089] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:43:11.158164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123a5ed-5403-4307-84bf-bcc28a0607b9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:11.159817 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4123a5ed-5403-4307-84bf-bcc28a0607b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:11.172371 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123a5ed-5403-4307-84bf-bcc28a0607b9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:11.177845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874a4ee-68d1-43c7-9736-f7555c27078f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:11.178549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874a4ee-68d1-43c7-9736-f7555c27078f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:11.178858 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874a4ee-68d1-43c7-9736-f7555c27078f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:11.179274 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874a4ee-68d1-43c7-9736-f7555c27078f admin admin] PATH_INFO: `/v3/projects/53df71b1486e462291225fa8f39ea8f5` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:11.180465 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874a4ee-68d1-43c7-9736-f7555c27078f admin admin] RBAC: Authorizing `identity:delete_project(project_id=53df71b1486e462291225fa8f39ea8f5)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:11.182575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874a4ee-68d1-43c7-9736-f7555c27078f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:11.317109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3874a4ee-68d1-43c7-9736-f7555c27078f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53df71b1486e462291225fa8f39ea8f5', 'request_id': 'req-3874a4ee-68d1-43c7-9736-f7555c27078f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:11.322635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3874a4ee-68d1-43c7-9736-f7555c27078f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53df71b1486e462291225fa8f39ea8f5', 'request_id': 'req-3874a4ee-68d1-43c7-9736-f7555c27078f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:11.406464 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3874a4ee-68d1-43c7-9736-f7555c27078f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53df71b1486e462291225fa8f39ea8f5', 'request_id': 'req-3874a4ee-68d1-43c7-9736-f7555c27078f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:11.418050 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3874a4ee-68d1-43c7-9736-f7555c27078f admin admin] The token cache is being invalidate because project 53df71b1486e462291225fa8f39ea8f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:43:11.418401 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3874a4ee-68d1-43c7-9736-f7555c27078f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:11.418997 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3874a4ee-68d1-43c7-9736-f7555c27078f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:11.420509 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1036/2090] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:43:11 2022] DELETE /identity/v3/projects/53df71b1486e462291225fa8f39ea8f5 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:11.424656 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05781ba5-4142-496c-b091-c3919edb9e98 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:11.435078 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05781ba5-4142-496c-b091-c3919edb9e98 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:11.490915 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54a3bf-cfcf-4761-8565-f29654129fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:11.491596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54a3bf-cfcf-4761-8565-f29654129fb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:11.491876 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54a3bf-cfcf-4761-8565-f29654129fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:11.492151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54a3bf-cfcf-4761-8565-f29654129fb4 admin admin] PATH_INFO: `/v3/users/011a7112c6624136b30928a1b52a63b8` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:11.493263 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54a3bf-cfcf-4761-8565-f29654129fb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=011a7112c6624136b30928a1b52a63b8)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:11.495191 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54a3bf-cfcf-4761-8565-f29654129fb4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:11.690709 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ab54a3bf-cfcf-4761-8565-f29654129fb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '011a7112c6624136b30928a1b52a63b8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:11.697523 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ab54a3bf-cfcf-4761-8565-f29654129fb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '011a7112c6624136b30928a1b52a63b8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:11.766625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ab54a3bf-cfcf-4761-8565-f29654129fb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '011a7112c6624136b30928a1b52a63b8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:11.767437 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ab54a3bf-cfcf-4761-8565-f29654129fb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '011a7112c6624136b30928a1b52a63b8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:11.780507 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ab54a3bf-cfcf-4761-8565-f29654129fb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '011a7112c6624136b30928a1b52a63b8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:11.787628 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1055/2091] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:43:11 2022] DELETE /identity/v3/users/011a7112c6624136b30928a1b52a63b8 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:12.261695 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874a4ee-68d1-43c7-9736-f7555c27078f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:12.264002 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3874a4ee-68d1-43c7-9736-f7555c27078f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:12.284198 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874a4ee-68d1-43c7-9736-f7555c27078f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:12.347378 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a0480-67fa-491d-98c2-7f5bc1786c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:12.347378 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a0480-67fa-491d-98c2-7f5bc1786c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:12.347378 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a0480-67fa-491d-98c2-7f5bc1786c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:12.347738 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a0480-67fa-491d-98c2-7f5bc1786c38 admin admin] PATH_INFO: `/v3/projects/266cf62ea24445608fd4914eb86e381a` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:12.348611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26a0480-67fa-491d-98c2-7f5bc1786c38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=266cf62ea24445608fd4914eb86e381a)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:12.350521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26a0480-67fa-491d-98c2-7f5bc1786c38 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:12.463461 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b26a0480-67fa-491d-98c2-7f5bc1786c38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '266cf62ea24445608fd4914eb86e381a', 'request_id': 'req-b26a0480-67fa-491d-98c2-7f5bc1786c38'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:12.470599 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b26a0480-67fa-491d-98c2-7f5bc1786c38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '266cf62ea24445608fd4914eb86e381a', 'request_id': 'req-b26a0480-67fa-491d-98c2-7f5bc1786c38'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:12.473706 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:12.561593 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b26a0480-67fa-491d-98c2-7f5bc1786c38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '266cf62ea24445608fd4914eb86e381a', 'request_id': 'req-b26a0480-67fa-491d-98c2-7f5bc1786c38'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:12.573854 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b26a0480-67fa-491d-98c2-7f5bc1786c38 admin admin] The token cache is being invalidate because project 266cf62ea24445608fd4914eb86e381a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:43:12.574127 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b26a0480-67fa-491d-98c2-7f5bc1786c38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:12.574931 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b26a0480-67fa-491d-98c2-7f5bc1786c38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:12.577379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1037/2092] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:43:12 2022] DELETE /identity/v3/projects/266cf62ea24445608fd4914eb86e381a => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:43:12.582346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64eb8ee-65ac-4359-822b-70705ec203a5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:12.587443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64eb8ee-65ac-4359-822b-70705ec203a5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:12.587443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64eb8ee-65ac-4359-822b-70705ec203a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:12.603392 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c64eb8ee-65ac-4359-822b-70705ec203a5 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:12.604604 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c64eb8ee-65ac-4359-822b-70705ec203a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:12.649751 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:12.664895 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1056/2093] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:12.672620 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea669c-7c20-46c5-9aa8-5903b498d72f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:12.672905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea669c-7c20-46c5-9aa8-5903b498d72f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:12.673172 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea669c-7c20-46c5-9aa8-5903b498d72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:12.692067 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-29ea669c-7c20-46c5-9aa8-5903b498d72f None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:12.693261 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-29ea669c-7c20-46c5-9aa8-5903b498d72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:12.701788 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1038/2094] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:12.706056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64eb8ee-65ac-4359-822b-70705ec203a5 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:12.715963 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64eb8ee-65ac-4359-822b-70705ec203a5 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:12.720836 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50decc79-aac3-41ff-a7a7-4914bf751a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:12.721435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50decc79-aac3-41ff-a7a7-4914bf751a69 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:12.721718 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50decc79-aac3-41ff-a7a7-4914bf751a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:12.721990 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50decc79-aac3-41ff-a7a7-4914bf751a69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:12.722629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50decc79-aac3-41ff-a7a7-4914bf751a69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:43:12.722934 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50decc79-aac3-41ff-a7a7-4914bf751a69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:12.724572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50decc79-aac3-41ff-a7a7-4914bf751a69 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:12.736872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1057/2095] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:43:12.743533 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea669c-7c20-46c5-9aa8-5903b498d72f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:12.751055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea669c-7c20-46c5-9aa8-5903b498d72f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:12.756001 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c3795-7094-48f1-9433-6adfeda359d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:12.756614 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c3795-7094-48f1-9433-6adfeda359d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:12.756901 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c3795-7094-48f1-9433-6adfeda359d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:12.757176 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c3795-7094-48f1-9433-6adfeda359d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:12.757817 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c3795-7094-48f1-9433-6adfeda359d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:12.759962 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c3795-7094-48f1-9433-6adfeda359d6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:12.855506 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1039/2096] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:43:12 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:43:12.860909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50decc79-aac3-41ff-a7a7-4914bf751a69 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:12.871514 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50decc79-aac3-41ff-a7a7-4914bf751a69 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:12.929638 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209101be-4de3-4f62-9c22-9ddd8408b434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:12.930291 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209101be-4de3-4f62-9c22-9ddd8408b434 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:12.930575 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209101be-4de3-4f62-9c22-9ddd8408b434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:12.930852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209101be-4de3-4f62-9c22-9ddd8408b434 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:12.931523 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209101be-4de3-4f62-9c22-9ddd8408b434 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:12.933675 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209101be-4de3-4f62-9c22-9ddd8408b434 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:13.064526 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1058/2097] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:43:12 2022] POST /identity/v3/users => generated 293 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:43:13.073773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c3795-7094-48f1-9433-6adfeda359d6 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:13.085042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c3795-7094-48f1-9433-6adfeda359d6 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:13.091305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0e856-71d1-4fb2-b9fc-204c29d3cc0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:13.091603 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0e856-71d1-4fb2-b9fc-204c29d3cc0e admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:13.091819 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0e856-71d1-4fb2-b9fc-204c29d3cc0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:13.091969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0e856-71d1-4fb2-b9fc-204c29d3cc0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:13.092737 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b0e856-71d1-4fb2-b9fc-204c29d3cc0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:13.095862 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b0e856-71d1-4fb2-b9fc-204c29d3cc0e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:13.104544 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1040/2098] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:43:13.112058 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209101be-4de3-4f62-9c22-9ddd8408b434 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:13.122627 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209101be-4de3-4f62-9c22-9ddd8408b434 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:13.127804 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04ed12-7388-47f4-bc1f-b641f32a219a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:13.128407 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04ed12-7388-47f4-bc1f-b641f32a219a admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:13.128690 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04ed12-7388-47f4-bc1f-b641f32a219a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:13.128968 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04ed12-7388-47f4-bc1f-b641f32a219a admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:13.129618 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04ed12-7388-47f4-bc1f-b641f32a219a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:13.131206 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04ed12-7388-47f4-bc1f-b641f32a219a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:13.143102 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1059/2099] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:43:13.147538 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0e856-71d1-4fb2-b9fc-204c29d3cc0e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:13.158665 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0e856-71d1-4fb2-b9fc-204c29d3cc0e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:13.163552 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc2f70-b40e-4346-8f61-3ee4424c50b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:13.164205 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc2f70-b40e-4346-8f61-3ee4424c50b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:13.164489 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc2f70-b40e-4346-8f61-3ee4424c50b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:13.164757 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc2f70-b40e-4346-8f61-3ee4424c50b3 admin admin] PATH_INFO: `/v3/projects/273c9d834b454f9ab18d26ed0505a5a6/users/d3928386d8db4787855554241c8b34a8/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:13.181464 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc2f70-b40e-4346-8f61-3ee4424c50b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=273c9d834b454f9ab18d26ed0505a5a6, user_id=d3928386d8db4787855554241c8b34a8, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:13.184244 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc2f70-b40e-4346-8f61-3ee4424c50b3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:13.260210 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1041/2100] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:43:13 2022] PUT /identity/v3/projects/273c9d834b454f9ab18d26ed0505a5a6/users/d3928386d8db4787855554241c8b34a8/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:13.271411 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc75f42-a3ee-433c-b2a3-af1a9079c6b1 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:13.271697 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc75f42-a3ee-433c-b2a3-af1a9079c6b1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:13.271971 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc75f42-a3ee-433c-b2a3-af1a9079c6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:13.300762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9fc75f42-a3ee-433c-b2a3-af1a9079c6b1 None None] MFA Rules not processed for user `d3928386d8db4787855554241c8b34a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:13.302030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9fc75f42-a3ee-433c-b2a3-af1a9079c6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:13.365566 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1060/2101] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:13.375344 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc2f70-b40e-4346-8f61-3ee4424c50b3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:13.440158 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc2f70-b40e-4346-8f61-3ee4424c50b3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:13.444840 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e1fbce-a2be-4535-ab37-281b6436c2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:13.445428 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e1fbce-a2be-4535-ab37-281b6436c2b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:13.445714 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e1fbce-a2be-4535-ab37-281b6436c2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:13.445986 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e1fbce-a2be-4535-ab37-281b6436c2b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:13.446610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e1fbce-a2be-4535-ab37-281b6436c2b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:13.448540 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e1fbce-a2be-4535-ab37-281b6436c2b5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:13.535368 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1042/2102] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:43:13 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:43:13.539696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc75f42-a3ee-433c-b2a3-af1a9079c6b1 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:13.551232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc75f42-a3ee-433c-b2a3-af1a9079c6b1 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:13.596001 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:13.609184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759609f-66f1-4d3e-9852-f26a59855e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:13.609794 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759609f-66f1-4d3e-9852-f26a59855e0d admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:13.610080 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759609f-66f1-4d3e-9852-f26a59855e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:13.610358 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759609f-66f1-4d3e-9852-f26a59855e0d admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:13.610977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759609f-66f1-4d3e-9852-f26a59855e0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:13.613240 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759609f-66f1-4d3e-9852-f26a59855e0d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:13.791924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1061/2103] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:43:13 2022] POST /identity/v3/users => generated 299 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:43:13.796331 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e1fbce-a2be-4535-ab37-281b6436c2b5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:13.806644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e1fbce-a2be-4535-ab37-281b6436c2b5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:13.811200 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc070c51-e189-4a26-8239-931c8ed39265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:13.811799 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc070c51-e189-4a26-8239-931c8ed39265 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:13.812080 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc070c51-e189-4a26-8239-931c8ed39265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:13.812430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc070c51-e189-4a26-8239-931c8ed39265 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:13.813039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc070c51-e189-4a26-8239-931c8ed39265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:13.814631 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc070c51-e189-4a26-8239-931c8ed39265 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:13.825228 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1043/2104] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:43:13.829368 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759609f-66f1-4d3e-9852-f26a59855e0d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:13.839286 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759609f-66f1-4d3e-9852-f26a59855e0d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:13.844464 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a86bf29-929c-41c2-ba42-4981c1c28b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:13.845110 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a86bf29-929c-41c2-ba42-4981c1c28b63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:13.845388 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a86bf29-929c-41c2-ba42-4981c1c28b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:13.845662 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a86bf29-929c-41c2-ba42-4981c1c28b63 admin admin] PATH_INFO: `/v3/projects/5d744baa9e8a47bfbfb8bce372fa1a79/users/a5b992cbb54341a78b876a6d372d3435/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:13.861052 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a86bf29-929c-41c2-ba42-4981c1c28b63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d744baa9e8a47bfbfb8bce372fa1a79, user_id=a5b992cbb54341a78b876a6d372d3435, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:13.863982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a86bf29-929c-41c2-ba42-4981c1c28b63 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:13.934095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1062/2105] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:43:13 2022] PUT /identity/v3/projects/5d744baa9e8a47bfbfb8bce372fa1a79/users/a5b992cbb54341a78b876a6d372d3435/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:13.938036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc070c51-e189-4a26-8239-931c8ed39265 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:13.948023 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc070c51-e189-4a26-8239-931c8ed39265 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:14.009943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21dc626-6f33-498d-a583-0f0de9b7b05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:14.010559 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21dc626-6f33-498d-a583-0f0de9b7b05d admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:14.010836 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21dc626-6f33-498d-a583-0f0de9b7b05d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:14.011104 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21dc626-6f33-498d-a583-0f0de9b7b05d admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:14.011773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21dc626-6f33-498d-a583-0f0de9b7b05d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:14.014718 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21dc626-6f33-498d-a583-0f0de9b7b05d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:14.016183 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:14.029354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1044/2106] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:43:14.031030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a86bf29-929c-41c2-ba42-4981c1c28b63 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:14.057429 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a86bf29-929c-41c2-ba42-4981c1c28b63 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:14.065521 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1f998-1206-44ba-8169-5f69f2509802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:14.066208 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1f998-1206-44ba-8169-5f69f2509802 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:14.066490 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1f998-1206-44ba-8169-5f69f2509802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:14.066769 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1f998-1206-44ba-8169-5f69f2509802 admin admin] PATH_INFO: `/v3/projects/5d744baa9e8a47bfbfb8bce372fa1a79/users/a5b992cbb54341a78b876a6d372d3435/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:14.068821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1f998-1206-44ba-8169-5f69f2509802 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d744baa9e8a47bfbfb8bce372fa1a79, user_id=a5b992cbb54341a78b876a6d372d3435, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:14.072039 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1f998-1206-44ba-8169-5f69f2509802 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:14.083752 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1063/2107] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:43:14 2022] PUT /identity/v3/projects/5d744baa9e8a47bfbfb8bce372fa1a79/users/a5b992cbb54341a78b876a6d372d3435/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:14.099362 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cee117-5690-43b7-957d-fb943a1adf46 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:14.099362 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cee117-5690-43b7-957d-fb943a1adf46 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:14.099362 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cee117-5690-43b7-957d-fb943a1adf46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:14.129751 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-22cee117-5690-43b7-957d-fb943a1adf46 None None] MFA Rules not processed for user `a5b992cbb54341a78b876a6d372d3435`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:14.131108 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-22cee117-5690-43b7-957d-fb943a1adf46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:14.205559 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1045/2108] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:14 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:14.213253 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1f998-1206-44ba-8169-5f69f2509802 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:14.225177 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-51d1f998-1206-44ba-8169-5f69f2509802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:14.277220 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1f998-1206-44ba-8169-5f69f2509802 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:14.282072 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65935222-710b-4079-b0c1-d18990976f8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:14.282989 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65935222-710b-4079-b0c1-d18990976f8a service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:14.283377 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65935222-710b-4079-b0c1-d18990976f8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:14.283725 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65935222-710b-4079-b0c1-d18990976f8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:14.297180 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65935222-710b-4079-b0c1-d18990976f8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:14.304131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65935222-710b-4079-b0c1-d18990976f8a service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:14.310837 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1064/2109] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:14 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:43:15.496383 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cee117-5690-43b7-957d-fb943a1adf46 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:15.558998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cee117-5690-43b7-957d-fb943a1adf46 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:15.563786 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a7ec3-fd7d-4fb1-a5a9-1f39293f213b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:15.564406 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a7ec3-fd7d-4fb1-a5a9-1f39293f213b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:15.564687 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a7ec3-fd7d-4fb1-a5a9-1f39293f213b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:15.564965 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a7ec3-fd7d-4fb1-a5a9-1f39293f213b admin admin] PATH_INFO: `/v3/users/d3928386d8db4787855554241c8b34a8` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:15.566078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a7ec3-fd7d-4fb1-a5a9-1f39293f213b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3928386d8db4787855554241c8b34a8)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:15.568008 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a7ec3-fd7d-4fb1-a5a9-1f39293f213b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:15.775580 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-828a7ec3-fd7d-4fb1-a5a9-1f39293f213b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3928386d8db4787855554241c8b34a8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:15.847475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-828a7ec3-fd7d-4fb1-a5a9-1f39293f213b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3928386d8db4787855554241c8b34a8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:15.848045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-828a7ec3-fd7d-4fb1-a5a9-1f39293f213b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3928386d8db4787855554241c8b34a8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:15.848640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-828a7ec3-fd7d-4fb1-a5a9-1f39293f213b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3928386d8db4787855554241c8b34a8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:15.855803 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-828a7ec3-fd7d-4fb1-a5a9-1f39293f213b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3928386d8db4787855554241c8b34a8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:15.869890 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:15.874952 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1046/2110] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:43:15 2022] DELETE /identity/v3/users/d3928386d8db4787855554241c8b34a8 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:15.879455 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26f7c4-ecee-4dd4-abab-31451a12bc36 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:15.879930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26f7c4-ecee-4dd4-abab-31451a12bc36 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:15.880086 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26f7c4-ecee-4dd4-abab-31451a12bc36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:15.899238 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-fa26f7c4-ecee-4dd4-abab-31451a12bc36 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:15.900450 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-fa26f7c4-ecee-4dd4-abab-31451a12bc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:15.960957 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1065/2111] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:15.971107 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a7ec3-fd7d-4fb1-a5a9-1f39293f213b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:15.972677 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-828a7ec3-fd7d-4fb1-a5a9-1f39293f213b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:16.030043 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a7ec3-fd7d-4fb1-a5a9-1f39293f213b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:16.033774 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9404b4-05c5-418b-baa5-b8f4c0c8e341 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:16.034381 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9404b4-05c5-418b-baa5-b8f4c0c8e341 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:16.034661 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9404b4-05c5-418b-baa5-b8f4c0c8e341 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:16.034945 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9404b4-05c5-418b-baa5-b8f4c0c8e341 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:16.044760 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9404b4-05c5-418b-baa5-b8f4c0c8e341 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:16.046187 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9404b4-05c5-418b-baa5-b8f4c0c8e341 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:16.055426 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1047/2112] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:43:16.473491 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa26f7c4-ecee-4dd4-abab-31451a12bc36 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:16.475355 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-fa26f7c4-ecee-4dd4-abab-31451a12bc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:16.490419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa26f7c4-ecee-4dd4-abab-31451a12bc36 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:16.496063 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56bfe69-08fd-40b8-8e31-a3fdec6fc947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:16.496708 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bfe69-08fd-40b8-8e31-a3fdec6fc947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:16.496993 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bfe69-08fd-40b8-8e31-a3fdec6fc947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:16.497324 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bfe69-08fd-40b8-8e31-a3fdec6fc947 admin admin] PATH_INFO: `/v3/projects/273c9d834b454f9ab18d26ed0505a5a6` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:16.498755 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56bfe69-08fd-40b8-8e31-a3fdec6fc947 admin admin] RBAC: Authorizing `identity:delete_project(project_id=273c9d834b454f9ab18d26ed0505a5a6)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:16.510428 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56bfe69-08fd-40b8-8e31-a3fdec6fc947 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:16.634436 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:16.643965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a56bfe69-08fd-40b8-8e31-a3fdec6fc947 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '273c9d834b454f9ab18d26ed0505a5a6', 'request_id': 'req-a56bfe69-08fd-40b8-8e31-a3fdec6fc947'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:16.649394 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a56bfe69-08fd-40b8-8e31-a3fdec6fc947 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '273c9d834b454f9ab18d26ed0505a5a6', 'request_id': 'req-a56bfe69-08fd-40b8-8e31-a3fdec6fc947'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:16.743741 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a56bfe69-08fd-40b8-8e31-a3fdec6fc947 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '273c9d834b454f9ab18d26ed0505a5a6', 'request_id': 'req-a56bfe69-08fd-40b8-8e31-a3fdec6fc947'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:16.755966 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a56bfe69-08fd-40b8-8e31-a3fdec6fc947 admin admin] The token cache is being invalidate because project 273c9d834b454f9ab18d26ed0505a5a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:43:16.756450 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a56bfe69-08fd-40b8-8e31-a3fdec6fc947 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:16.757607 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a56bfe69-08fd-40b8-8e31-a3fdec6fc947 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:16.759514 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1066/2113] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:43:16 2022] DELETE /identity/v3/projects/273c9d834b454f9ab18d26ed0505a5a6 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:16.764665 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9404b4-05c5-418b-baa5-b8f4c0c8e341 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:16.774946 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9404b4-05c5-418b-baa5-b8f4c0c8e341 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:16.832851 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b005e39f-077e-4a2c-b021-8d35fb9e3528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:16.833487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005e39f-077e-4a2c-b021-8d35fb9e3528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:16.833769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005e39f-077e-4a2c-b021-8d35fb9e3528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:16.834045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005e39f-077e-4a2c-b021-8d35fb9e3528 admin admin] PATH_INFO: `/v3/users/a5b992cbb54341a78b876a6d372d3435` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:16.835198 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b005e39f-077e-4a2c-b021-8d35fb9e3528 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5b992cbb54341a78b876a6d372d3435)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:16.837127 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b005e39f-077e-4a2c-b021-8d35fb9e3528 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:17.040656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b005e39f-077e-4a2c-b021-8d35fb9e3528 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5b992cbb54341a78b876a6d372d3435'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:17.099506 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b005e39f-077e-4a2c-b021-8d35fb9e3528 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5b992cbb54341a78b876a6d372d3435'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:17.100126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b005e39f-077e-4a2c-b021-8d35fb9e3528 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5b992cbb54341a78b876a6d372d3435'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:17.100748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b005e39f-077e-4a2c-b021-8d35fb9e3528 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5b992cbb54341a78b876a6d372d3435'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:17.109153 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b005e39f-077e-4a2c-b021-8d35fb9e3528 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5b992cbb54341a78b876a6d372d3435'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:17.127947 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1048/2114] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:43:16 2022] DELETE /identity/v3/users/a5b992cbb54341a78b876a6d372d3435 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:17.594830 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56bfe69-08fd-40b8-8e31-a3fdec6fc947 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:17.597077 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a56bfe69-08fd-40b8-8e31-a3fdec6fc947 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:17.621441 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56bfe69-08fd-40b8-8e31-a3fdec6fc947 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:17.720020 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d033f3-28eb-4a0b-9dd3-662311df7231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:17.720849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d033f3-28eb-4a0b-9dd3-662311df7231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:17.721421 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d033f3-28eb-4a0b-9dd3-662311df7231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:17.721699 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d033f3-28eb-4a0b-9dd3-662311df7231 admin admin] PATH_INFO: `/v3/projects/5d744baa9e8a47bfbfb8bce372fa1a79` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:17.722851 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d033f3-28eb-4a0b-9dd3-662311df7231 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d744baa9e8a47bfbfb8bce372fa1a79)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:17.724882 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d033f3-28eb-4a0b-9dd3-662311df7231 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:17.842551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d8d033f3-28eb-4a0b-9dd3-662311df7231 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d744baa9e8a47bfbfb8bce372fa1a79', 'request_id': 'req-d8d033f3-28eb-4a0b-9dd3-662311df7231'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:17.848204 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d8d033f3-28eb-4a0b-9dd3-662311df7231 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d744baa9e8a47bfbfb8bce372fa1a79', 'request_id': 'req-d8d033f3-28eb-4a0b-9dd3-662311df7231'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:17.990530 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d8d033f3-28eb-4a0b-9dd3-662311df7231 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d744baa9e8a47bfbfb8bce372fa1a79', 'request_id': 'req-d8d033f3-28eb-4a0b-9dd3-662311df7231'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:18.002759 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d8d033f3-28eb-4a0b-9dd3-662311df7231 admin admin] The token cache is being invalidate because project 5d744baa9e8a47bfbfb8bce372fa1a79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:43:18.003054 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d8d033f3-28eb-4a0b-9dd3-662311df7231 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:18.003734 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d8d033f3-28eb-4a0b-9dd3-662311df7231 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:18.005374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1067/2115] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:43:17 2022] DELETE /identity/v3/projects/5d744baa9e8a47bfbfb8bce372fa1a79 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:18.010246 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716d0a4-7578-42b3-82a4-5bae09788db1 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:18.010708 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716d0a4-7578-42b3-82a4-5bae09788db1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:18.010803 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716d0a4-7578-42b3-82a4-5bae09788db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:18.033161 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4716d0a4-7578-42b3-82a4-5bae09788db1 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:18.034415 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4716d0a4-7578-42b3-82a4-5bae09788db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:18.052842 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:18.097491 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1049/2116] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:18.105922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c89921-7767-4cfc-b263-bfa928aa90f8 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:18.106216 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c89921-7767-4cfc-b263-bfa928aa90f8 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:18.106513 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c89921-7767-4cfc-b263-bfa928aa90f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:18.132395 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b7c89921-7767-4cfc-b263-bfa928aa90f8 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:18.133681 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b7c89921-7767-4cfc-b263-bfa928aa90f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:18.143206 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1068/2117] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:18.146944 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716d0a4-7578-42b3-82a4-5bae09788db1 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:18.157416 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716d0a4-7578-42b3-82a4-5bae09788db1 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:18.162134 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d84d0e-8c6f-4320-9adf-886362924265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:18.162727 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d84d0e-8c6f-4320-9adf-886362924265 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:18.163010 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d84d0e-8c6f-4320-9adf-886362924265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:18.163326 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d84d0e-8c6f-4320-9adf-886362924265 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:18.163974 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d84d0e-8c6f-4320-9adf-886362924265 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:43:18.164284 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d84d0e-8c6f-4320-9adf-886362924265 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:18.165923 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d84d0e-8c6f-4320-9adf-886362924265 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:18.178728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1050/2118] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:43:18.183239 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c89921-7767-4cfc-b263-bfa928aa90f8 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:18.193192 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c89921-7767-4cfc-b263-bfa928aa90f8 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:18.200058 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8327e0-20dd-46c6-af2f-a0c9f7bdca4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:18.200058 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8327e0-20dd-46c6-af2f-a0c9f7bdca4e admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:18.200350 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8327e0-20dd-46c6-af2f-a0c9f7bdca4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:18.200696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8327e0-20dd-46c6-af2f-a0c9f7bdca4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:18.201530 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8327e0-20dd-46c6-af2f-a0c9f7bdca4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:18.205139 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8327e0-20dd-46c6-af2f-a0c9f7bdca4e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:18.331664 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1069/2119] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:43:18 2022] POST /identity/v3/projects => generated 373 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:43:18.340111 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d84d0e-8c6f-4320-9adf-886362924265 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:18.352172 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d84d0e-8c6f-4320-9adf-886362924265 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:18.419016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b24a72-39de-49c0-b9ce-362499047dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:18.420890 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b24a72-39de-49c0-b9ce-362499047dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:18.421326 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b24a72-39de-49c0-b9ce-362499047dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:18.421720 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b24a72-39de-49c0-b9ce-362499047dd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:18.422444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b24a72-39de-49c0-b9ce-362499047dd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:18.424664 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b24a72-39de-49c0-b9ce-362499047dd8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:18.511630 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1051/2120] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:43:18 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:43:18.516495 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8327e0-20dd-46c6-af2f-a0c9f7bdca4e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:18.527107 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8327e0-20dd-46c6-af2f-a0c9f7bdca4e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:18.532493 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58edfc-d4be-4684-9a7a-644d2f1769b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:18.533090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58edfc-d4be-4684-9a7a-644d2f1769b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:18.533365 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58edfc-d4be-4684-9a7a-644d2f1769b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:18.533637 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58edfc-d4be-4684-9a7a-644d2f1769b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:18.534267 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a58edfc-d4be-4684-9a7a-644d2f1769b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:18.535916 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a58edfc-d4be-4684-9a7a-644d2f1769b8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:18.550046 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1070/2121] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:43:18.554610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b24a72-39de-49c0-b9ce-362499047dd8 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:18.565553 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b24a72-39de-49c0-b9ce-362499047dd8 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:18.570363 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9dcb16-16ed-4ea0-baf3-625ec9e165e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:18.571013 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9dcb16-16ed-4ea0-baf3-625ec9e165e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:18.571316 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9dcb16-16ed-4ea0-baf3-625ec9e165e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:18.571592 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9dcb16-16ed-4ea0-baf3-625ec9e165e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:18.572268 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9dcb16-16ed-4ea0-baf3-625ec9e165e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:18.573863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9dcb16-16ed-4ea0-baf3-625ec9e165e6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:18.585314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1052/2122] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:43:18.589555 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58edfc-d4be-4684-9a7a-644d2f1769b8 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:18.601058 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58edfc-d4be-4684-9a7a-644d2f1769b8 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:18.606078 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5de0fb-1b74-4a59-87bf-fc85449a7d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:18.606757 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5de0fb-1b74-4a59-87bf-fc85449a7d1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:18.607589 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5de0fb-1b74-4a59-87bf-fc85449a7d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:18.607874 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5de0fb-1b74-4a59-87bf-fc85449a7d1a admin admin] PATH_INFO: `/v3/projects/ca783a603c484aad86d05156d7e9142c/users/00b19e56e11047e3b0f102efea439b7f/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:18.620908 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:18.624352 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5de0fb-1b74-4a59-87bf-fc85449a7d1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca783a603c484aad86d05156d7e9142c, user_id=00b19e56e11047e3b0f102efea439b7f, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:18.627007 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5de0fb-1b74-4a59-87bf-fc85449a7d1a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:18.699577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1071/2123] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:43:18 2022] PUT /identity/v3/projects/ca783a603c484aad86d05156d7e9142c/users/00b19e56e11047e3b0f102efea439b7f/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:18.705383 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09de3e-49d7-413c-b8f7-166eddd65747 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:18.705668 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09de3e-49d7-413c-b8f7-166eddd65747 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:18.705969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09de3e-49d7-413c-b8f7-166eddd65747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:18.725931 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4e09de3e-49d7-413c-b8f7-166eddd65747 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:18.727150 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4e09de3e-49d7-413c-b8f7-166eddd65747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:18.788861 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1053/2124] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:18.799489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5de0fb-1b74-4a59-87bf-fc85449a7d1a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:18.801872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4a5de0fb-1b74-4a59-87bf-fc85449a7d1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:18.866246 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5de0fb-1b74-4a59-87bf-fc85449a7d1a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:18.870092 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c77e4-86fd-44ee-9023-700c8308f4bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:18.870850 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c77e4-86fd-44ee-9023-700c8308f4bd service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:18.871217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c77e4-86fd-44ee-9023-700c8308f4bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:18.871615 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c77e4-86fd-44ee-9023-700c8308f4bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:18.881762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c77e4-86fd-44ee-9023-700c8308f4bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:18.884158 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c77e4-86fd-44ee-9023-700c8308f4bd service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:18.894550 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1072/2125] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:43:20.312678 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448a93e-407e-40d4-ab5f-ed71b2c6fde6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:20.313195 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448a93e-407e-40d4-ab5f-ed71b2c6fde6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:20.313468 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448a93e-407e-40d4-ab5f-ed71b2c6fde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:20.343049 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e448a93e-407e-40d4-ab5f-ed71b2c6fde6 None None] MFA Rules not processed for user `00b19e56e11047e3b0f102efea439b7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:20.344329 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e448a93e-407e-40d4-ab5f-ed71b2c6fde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:20.362302 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:20.423456 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1054/2126] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:20.483376 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c77e4-86fd-44ee-9023-700c8308f4bd service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:20.484918 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5b8c77e4-86fd-44ee-9023-700c8308f4bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:20.549228 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c77e4-86fd-44ee-9023-700c8308f4bd service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:20.552793 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5871cc6-8c76-48a8-bb8c-b374bd0adba1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:20.553393 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5871cc6-8c76-48a8-bb8c-b374bd0adba1 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:20.553668 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5871cc6-8c76-48a8-bb8c-b374bd0adba1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:20.553939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5871cc6-8c76-48a8-bb8c-b374bd0adba1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:20.562749 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5871cc6-8c76-48a8-bb8c-b374bd0adba1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:20.564324 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5871cc6-8c76-48a8-bb8c-b374bd0adba1 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:20.573187 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1073/2127] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:43:22.684405 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f1209-9dce-49f4-8897-564e4e1165b6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:22.684912 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f1209-9dce-49f4-8897-564e4e1165b6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:22.685002 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f1209-9dce-49f4-8897-564e4e1165b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:22.704653 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5f7f1209-9dce-49f4-8897-564e4e1165b6 None None] MFA Rules not processed for user `00b19e56e11047e3b0f102efea439b7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:22.705877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5f7f1209-9dce-49f4-8897-564e4e1165b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:22.713106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1055/2128] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:30.569055 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=55,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:32.389101 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5871cc6-8c76-48a8-bb8c-b374bd0adba1 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:32.391409 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d5871cc6-8c76-48a8-bb8c-b374bd0adba1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:32.405565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5871cc6-8c76-48a8-bb8c-b374bd0adba1 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:32.410415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd88b33-fabe-4327-a31e-f477c1bbc214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:32.411049 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd88b33-fabe-4327-a31e-f477c1bbc214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:32.411381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd88b33-fabe-4327-a31e-f477c1bbc214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:32.411689 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd88b33-fabe-4327-a31e-f477c1bbc214 admin admin] PATH_INFO: `/v3/users/0384636148ec4da5999e2cac62de2165` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:32.412861 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd88b33-fabe-4327-a31e-f477c1bbc214 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0384636148ec4da5999e2cac62de2165)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:32.414799 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd88b33-fabe-4327-a31e-f477c1bbc214 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:32.605194 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8dd88b33-fabe-4327-a31e-f477c1bbc214 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0384636148ec4da5999e2cac62de2165'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:32.612945 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8dd88b33-fabe-4327-a31e-f477c1bbc214 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0384636148ec4da5999e2cac62de2165'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:32.682387 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8dd88b33-fabe-4327-a31e-f477c1bbc214 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0384636148ec4da5999e2cac62de2165'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:32.683118 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8dd88b33-fabe-4327-a31e-f477c1bbc214 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0384636148ec4da5999e2cac62de2165'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:32.683991 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8dd88b33-fabe-4327-a31e-f477c1bbc214 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0384636148ec4da5999e2cac62de2165'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:32.700245 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=22 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:32.706877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1074/2129] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:43:32 2022] DELETE /identity/v3/users/0384636148ec4da5999e2cac62de2165 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:32.711432 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c37b25-4bf7-4f43-9bb9-626694a7d7b7 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:32.711718 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c37b25-4bf7-4f43-9bb9-626694a7d7b7 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:32.711991 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c37b25-4bf7-4f43-9bb9-626694a7d7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:32.733851 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d9c37b25-4bf7-4f43-9bb9-626694a7d7b7 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:32.740992 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d9c37b25-4bf7-4f43-9bb9-626694a7d7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:32.799225 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1056/2130] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:32.809167 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd88b33-fabe-4327-a31e-f477c1bbc214 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:32.810752 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8dd88b33-fabe-4327-a31e-f477c1bbc214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:32.879473 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd88b33-fabe-4327-a31e-f477c1bbc214 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:32.883227 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902c01b-9b8f-47b3-aaef-614283f5247f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:32.883855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902c01b-9b8f-47b3-aaef-614283f5247f service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:32.884135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902c01b-9b8f-47b3-aaef-614283f5247f service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:32.884411 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902c01b-9b8f-47b3-aaef-614283f5247f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:32.895107 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902c01b-9b8f-47b3-aaef-614283f5247f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:32.896598 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902c01b-9b8f-47b3-aaef-614283f5247f service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:32.909288 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1075/2131] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:43:33.012499 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e4258-9131-48bd-b81b-177e4e929fa4 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:33.012809 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e4258-9131-48bd-b81b-177e4e929fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:33.013086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e4258-9131-48bd-b81b-177e4e929fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:33.038390 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-2f8e4258-9131-48bd-b81b-177e4e929fa4 None None] MFA Rules not processed for user `39cb7e0c86b3460dbd17c58dc13ef0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:33.039673 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2f8e4258-9131-48bd-b81b-177e4e929fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:33.166466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1057/2132] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:33 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:33.361679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902c01b-9b8f-47b3-aaef-614283f5247f service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:33.364656 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8902c01b-9b8f-47b3-aaef-614283f5247f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:33.380132 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902c01b-9b8f-47b3-aaef-614283f5247f service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:33.385124 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04c83c-2d91-45ec-9820-ca14db63335e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:33.386109 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04c83c-2d91-45ec-9820-ca14db63335e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:33.387068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04c83c-2d91-45ec-9820-ca14db63335e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:33.387068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04c83c-2d91-45ec-9820-ca14db63335e admin admin] PATH_INFO: `/v3/projects/0d8eba4e4b1f4017b35f87797c09854d` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:33.388274 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04c83c-2d91-45ec-9820-ca14db63335e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d8eba4e4b1f4017b35f87797c09854d)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:33.391252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04c83c-2d91-45ec-9820-ca14db63335e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:33.513567 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2e04c83c-2d91-45ec-9820-ca14db63335e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d8eba4e4b1f4017b35f87797c09854d', 'request_id': 'req-2e04c83c-2d91-45ec-9820-ca14db63335e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:33.519113 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2e04c83c-2d91-45ec-9820-ca14db63335e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d8eba4e4b1f4017b35f87797c09854d', 'request_id': 'req-2e04c83c-2d91-45ec-9820-ca14db63335e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:33.629777 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2e04c83c-2d91-45ec-9820-ca14db63335e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d8eba4e4b1f4017b35f87797c09854d', 'request_id': 'req-2e04c83c-2d91-45ec-9820-ca14db63335e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:33.642414 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:33.647094 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2e04c83c-2d91-45ec-9820-ca14db63335e admin admin] The token cache is being invalidate because project 0d8eba4e4b1f4017b35f87797c09854d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:43:33.647467 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2e04c83c-2d91-45ec-9820-ca14db63335e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:33.648164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2e04c83c-2d91-45ec-9820-ca14db63335e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:33.649680 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1076/2133] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:43:33 2022] DELETE /identity/v3/projects/0d8eba4e4b1f4017b35f87797c09854d => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:33.653750 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e4258-9131-48bd-b81b-177e4e929fa4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:33.666449 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e4258-9131-48bd-b81b-177e4e929fa4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:33.729601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3139e47-65e5-43f0-9d8a-c1c614d25790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:33.730279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3139e47-65e5-43f0-9d8a-c1c614d25790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:33.730474 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3139e47-65e5-43f0-9d8a-c1c614d25790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:33.730696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3139e47-65e5-43f0-9d8a-c1c614d25790 admin admin] PATH_INFO: `/v3/users/c3a6d87ff1224b048a9fea81dfbe2e50` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:33.736389 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3139e47-65e5-43f0-9d8a-c1c614d25790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3a6d87ff1224b048a9fea81dfbe2e50)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:33.738656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3139e47-65e5-43f0-9d8a-c1c614d25790 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:33.862403 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d3139e47-65e5-43f0-9d8a-c1c614d25790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3a6d87ff1224b048a9fea81dfbe2e50'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:33.864129 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:33.951197 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d3139e47-65e5-43f0-9d8a-c1c614d25790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3a6d87ff1224b048a9fea81dfbe2e50'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:33.952453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d3139e47-65e5-43f0-9d8a-c1c614d25790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3a6d87ff1224b048a9fea81dfbe2e50'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:33.953077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d3139e47-65e5-43f0-9d8a-c1c614d25790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3a6d87ff1224b048a9fea81dfbe2e50'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:33.960920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d3139e47-65e5-43f0-9d8a-c1c614d25790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3a6d87ff1224b048a9fea81dfbe2e50'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:33.982035 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1058/2134] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:43:33 2022] DELETE /identity/v3/users/c3a6d87ff1224b048a9fea81dfbe2e50 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:34.506124 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04c83c-2d91-45ec-9820-ca14db63335e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:34.508874 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2e04c83c-2d91-45ec-9820-ca14db63335e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:34.524447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04c83c-2d91-45ec-9820-ca14db63335e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:34.589172 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d382e0-b6bd-4813-a10e-36f4f810ed83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:34.589838 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d382e0-b6bd-4813-a10e-36f4f810ed83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:34.590115 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d382e0-b6bd-4813-a10e-36f4f810ed83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:34.590403 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d382e0-b6bd-4813-a10e-36f4f810ed83 admin admin] PATH_INFO: `/v3/projects/f36052743f104c17b530d20339d1f1e1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:34.592183 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d382e0-b6bd-4813-a10e-36f4f810ed83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f36052743f104c17b530d20339d1f1e1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:34.594305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d382e0-b6bd-4813-a10e-36f4f810ed83 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:34.708409 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b3d382e0-b6bd-4813-a10e-36f4f810ed83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f36052743f104c17b530d20339d1f1e1', 'request_id': 'req-b3d382e0-b6bd-4813-a10e-36f4f810ed83'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:34.714093 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b3d382e0-b6bd-4813-a10e-36f4f810ed83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f36052743f104c17b530d20339d1f1e1', 'request_id': 'req-b3d382e0-b6bd-4813-a10e-36f4f810ed83'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:34.802746 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b3d382e0-b6bd-4813-a10e-36f4f810ed83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f36052743f104c17b530d20339d1f1e1', 'request_id': 'req-b3d382e0-b6bd-4813-a10e-36f4f810ed83'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:34.815561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b3d382e0-b6bd-4813-a10e-36f4f810ed83 admin admin] The token cache is being invalidate because project f36052743f104c17b530d20339d1f1e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:43:34.815852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b3d382e0-b6bd-4813-a10e-36f4f810ed83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:34.816507 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b3d382e0-b6bd-4813-a10e-36f4f810ed83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:34.818089 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1077/2135] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:43:34 2022] DELETE /identity/v3/projects/f36052743f104c17b530d20339d1f1e1 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:34.822922 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750a2bf-26a3-4822-bd65-01cebfd05683 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:34.823449 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750a2bf-26a3-4822-bd65-01cebfd05683 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:34.823551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750a2bf-26a3-4822-bd65-01cebfd05683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:34.844761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-2750a2bf-26a3-4822-bd65-01cebfd05683 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:34.845859 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2750a2bf-26a3-4822-bd65-01cebfd05683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:34.916410 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1059/2136] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:34.918414 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a0a05-ee8b-45a7-8042-9470d36312cb None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:34.918762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a0a05-ee8b-45a7-8042-9470d36312cb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:34.919072 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a0a05-ee8b-45a7-8042-9470d36312cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:34.941319 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7a9a0a05-ee8b-45a7-8042-9470d36312cb None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:34.943092 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7a9a0a05-ee8b-45a7-8042-9470d36312cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:34.959784 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1078/2137] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:34.961635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750a2bf-26a3-4822-bd65-01cebfd05683 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:34.977161 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750a2bf-26a3-4822-bd65-01cebfd05683 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:34.983896 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8a6a6-bc95-45df-8d46-5fa523517d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:34.984976 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8a6a6-bc95-45df-8d46-5fa523517d57 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:34.985370 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8a6a6-bc95-45df-8d46-5fa523517d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:34.985751 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8a6a6-bc95-45df-8d46-5fa523517d57 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:34.986759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa8a6a6-bc95-45df-8d46-5fa523517d57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:43:34.987149 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa8a6a6-bc95-45df-8d46-5fa523517d57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:34.988856 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa8a6a6-bc95-45df-8d46-5fa523517d57 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:35.008802 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1060/2138] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:43:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:43:35.013776 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a0a05-ee8b-45a7-8042-9470d36312cb None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:35.023202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a0a05-ee8b-45a7-8042-9470d36312cb None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:35.028314 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d4fde-932b-4fb4-a062-3e519ba1e543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:35.028926 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d4fde-932b-4fb4-a062-3e519ba1e543 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:35.029209 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d4fde-932b-4fb4-a062-3e519ba1e543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:35.029481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d4fde-932b-4fb4-a062-3e519ba1e543 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:35.030136 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d4fde-932b-4fb4-a062-3e519ba1e543 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:35.032352 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d4fde-932b-4fb4-a062-3e519ba1e543 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:35.152460 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1079/2139] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:43:35 2022] POST /identity/v3/projects => generated 387 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:43:35.156979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8a6a6-bc95-45df-8d46-5fa523517d57 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:35.175330 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8a6a6-bc95-45df-8d46-5fa523517d57 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:35.287301 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be536ce7-7455-4226-a6a8-b88c88a34fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:35.294515 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be536ce7-7455-4226-a6a8-b88c88a34fcf admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:35.295128 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be536ce7-7455-4226-a6a8-b88c88a34fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:35.295128 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be536ce7-7455-4226-a6a8-b88c88a34fcf admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:35.296409 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be536ce7-7455-4226-a6a8-b88c88a34fcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:35.301336 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be536ce7-7455-4226-a6a8-b88c88a34fcf admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:35.382068 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1061/2140] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:43:35 2022] POST /identity/v3/users => generated 294 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:43:35.386439 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d4fde-932b-4fb4-a062-3e519ba1e543 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:35.398997 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d4fde-932b-4fb4-a062-3e519ba1e543 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:35.403730 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce5930-e835-466a-a137-a79a92effd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:35.404424 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce5930-e835-466a-a137-a79a92effd63 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:35.404703 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce5930-e835-466a-a137-a79a92effd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:35.404971 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce5930-e835-466a-a137-a79a92effd63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:35.407536 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce5930-e835-466a-a137-a79a92effd63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:35.409644 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce5930-e835-466a-a137-a79a92effd63 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:35.425193 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1080/2141] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:43:35.433148 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be536ce7-7455-4226-a6a8-b88c88a34fcf admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:35.440504 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be536ce7-7455-4226-a6a8-b88c88a34fcf admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:35.451126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c200115-1c23-4311-ba50-ebc86e405391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:35.452067 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c200115-1c23-4311-ba50-ebc86e405391 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:35.453128 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c200115-1c23-4311-ba50-ebc86e405391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:35.453602 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c200115-1c23-4311-ba50-ebc86e405391 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:35.454471 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c200115-1c23-4311-ba50-ebc86e405391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:35.456339 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c200115-1c23-4311-ba50-ebc86e405391 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:35.468981 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1062/2142] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:43:35.478847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce5930-e835-466a-a137-a79a92effd63 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:35.492293 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce5930-e835-466a-a137-a79a92effd63 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:35.499706 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc9314-4df1-4926-9357-a769104aa413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:35.500401 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc9314-4df1-4926-9357-a769104aa413 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:35.500721 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc9314-4df1-4926-9357-a769104aa413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:35.500995 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc9314-4df1-4926-9357-a769104aa413 admin admin] PATH_INFO: `/v3/projects/56bd474411ba4c59a6fafbdc62313a17/users/3954a7b761664f9295983786a007cd56/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:35.521595 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc9314-4df1-4926-9357-a769104aa413 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56bd474411ba4c59a6fafbdc62313a17, user_id=3954a7b761664f9295983786a007cd56, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:35.524295 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc9314-4df1-4926-9357-a769104aa413 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:35.661219 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1081/2143] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:43:35 2022] PUT /identity/v3/projects/56bd474411ba4c59a6fafbdc62313a17/users/3954a7b761664f9295983786a007cd56/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:35.673740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da63aee-1280-4e30-9245-6bb9d08c90ae None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:35.674068 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da63aee-1280-4e30-9245-6bb9d08c90ae None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:35.674323 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da63aee-1280-4e30-9245-6bb9d08c90ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:35.701824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3da63aee-1280-4e30-9245-6bb9d08c90ae None None] MFA Rules not processed for user `3954a7b761664f9295983786a007cd56`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:35.703574 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3da63aee-1280-4e30-9245-6bb9d08c90ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:35.723027 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:35.771084 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1063/2144] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:43:35.775824 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc9314-4df1-4926-9357-a769104aa413 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:35.804262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:35.849074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc9314-4df1-4926-9357-a769104aa413 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:35.856747 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236336a-dfa6-44d8-84e4-109efe8d1003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:35.857370 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236336a-dfa6-44d8-84e4-109efe8d1003 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:35.857706 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236336a-dfa6-44d8-84e4-109efe8d1003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:35.857930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236336a-dfa6-44d8-84e4-109efe8d1003 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:35.859325 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5236336a-dfa6-44d8-84e4-109efe8d1003 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:35.861235 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5236336a-dfa6-44d8-84e4-109efe8d1003 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:35.953715 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1082/2145] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:43:35 2022] POST /identity/v3/projects => generated 389 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:43:35.957492 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da63aee-1280-4e30-9245-6bb9d08c90ae None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:35.973789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da63aee-1280-4e30-9245-6bb9d08c90ae None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:36.032973 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b893a52-dd4b-422b-a9c9-64d4821616b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:36.033591 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b893a52-dd4b-422b-a9c9-64d4821616b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:36.033875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b893a52-dd4b-422b-a9c9-64d4821616b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:36.034152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b893a52-dd4b-422b-a9c9-64d4821616b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:36.034789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b893a52-dd4b-422b-a9c9-64d4821616b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:36.036749 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b893a52-dd4b-422b-a9c9-64d4821616b3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:36.142167 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1064/2146] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:43:35 2022] POST /identity/v3/users => generated 301 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:43:36.146502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236336a-dfa6-44d8-84e4-109efe8d1003 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:36.157850 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236336a-dfa6-44d8-84e4-109efe8d1003 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:36.163272 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddcc6d8-4c07-4d4d-bbf3-bd94eaa2a298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:36.163880 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcc6d8-4c07-4d4d-bbf3-bd94eaa2a298 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:36.164161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcc6d8-4c07-4d4d-bbf3-bd94eaa2a298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:36.164436 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcc6d8-4c07-4d4d-bbf3-bd94eaa2a298 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:36.165107 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddcc6d8-4c07-4d4d-bbf3-bd94eaa2a298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:36.166684 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddcc6d8-4c07-4d4d-bbf3-bd94eaa2a298 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:36.178202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1083/2147] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:43:36.182738 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b893a52-dd4b-422b-a9c9-64d4821616b3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:36.194761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b893a52-dd4b-422b-a9c9-64d4821616b3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:36.203567 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0635c1c-ea89-46b0-bffb-52e093c384f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:36.203567 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0635c1c-ea89-46b0-bffb-52e093c384f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:36.203567 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0635c1c-ea89-46b0-bffb-52e093c384f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:36.204186 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0635c1c-ea89-46b0-bffb-52e093c384f2 admin admin] PATH_INFO: `/v3/projects/c629c2fd570f4b44bc6c5015890d8965/users/c77a6955f16d4e65bd519b877f1dea3e/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:36.223550 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0635c1c-ea89-46b0-bffb-52e093c384f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c629c2fd570f4b44bc6c5015890d8965, user_id=c77a6955f16d4e65bd519b877f1dea3e, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:36.226886 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0635c1c-ea89-46b0-bffb-52e093c384f2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:36.336910 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1065/2148] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:43:36 2022] PUT /identity/v3/projects/c629c2fd570f4b44bc6c5015890d8965/users/c77a6955f16d4e65bd519b877f1dea3e/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:36.341775 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddcc6d8-4c07-4d4d-bbf3-bd94eaa2a298 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:36.359396 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddcc6d8-4c07-4d4d-bbf3-bd94eaa2a298 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:36.441801 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7610348a-416e-43f5-97e4-9f9fb231ade1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:36.442435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610348a-416e-43f5-97e4-9f9fb231ade1 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:36.442943 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610348a-416e-43f5-97e4-9f9fb231ade1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:36.442943 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610348a-416e-43f5-97e4-9f9fb231ade1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:36.443635 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7610348a-416e-43f5-97e4-9f9fb231ade1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:36.445422 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7610348a-416e-43f5-97e4-9f9fb231ade1 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:36.460369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1084/2149] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:43:36.461608 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0635c1c-ea89-46b0-bffb-52e093c384f2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:36.511039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0635c1c-ea89-46b0-bffb-52e093c384f2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:36.532344 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e9969-d124-49dd-94b3-2ffaed92bdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:36.533059 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e9969-d124-49dd-94b3-2ffaed92bdb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:36.533387 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e9969-d124-49dd-94b3-2ffaed92bdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:36.533601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e9969-d124-49dd-94b3-2ffaed92bdb9 admin admin] PATH_INFO: `/v3/projects/c629c2fd570f4b44bc6c5015890d8965/users/c77a6955f16d4e65bd519b877f1dea3e/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:36.535660 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100e9969-d124-49dd-94b3-2ffaed92bdb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c629c2fd570f4b44bc6c5015890d8965, user_id=c77a6955f16d4e65bd519b877f1dea3e, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:36.538317 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100e9969-d124-49dd-94b3-2ffaed92bdb9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:36.552200 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1066/2150] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:43:36 2022] PUT /identity/v3/projects/c629c2fd570f4b44bc6c5015890d8965/users/c77a6955f16d4e65bd519b877f1dea3e/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:36.564596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cfddf-5c3d-41de-8b05-5eb4e3431134 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:36.565416 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cfddf-5c3d-41de-8b05-5eb4e3431134 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:36.565766 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cfddf-5c3d-41de-8b05-5eb4e3431134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:36.595789 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-631cfddf-5c3d-41de-8b05-5eb4e3431134 None None] MFA Rules not processed for user `c77a6955f16d4e65bd519b877f1dea3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:36.597138 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-631cfddf-5c3d-41de-8b05-5eb4e3431134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:36.685531 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1085/2151] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:36 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:43:36.702960 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e9969-d124-49dd-94b3-2ffaed92bdb9 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:36.705424 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-100e9969-d124-49dd-94b3-2ffaed92bdb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:36.781645 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e9969-d124-49dd-94b3-2ffaed92bdb9 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:36.786144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b632f47-a09b-4bae-b648-0100bccc02d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:36.786768 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b632f47-a09b-4bae-b648-0100bccc02d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:36.787049 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b632f47-a09b-4bae-b648-0100bccc02d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:36.787376 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b632f47-a09b-4bae-b648-0100bccc02d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:36.800355 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b632f47-a09b-4bae-b648-0100bccc02d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:36.802935 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b632f47-a09b-4bae-b648-0100bccc02d4 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:36.816914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1067/2152] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:36 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:43:36.999989 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631cfddf-5c3d-41de-8b05-5eb4e3431134 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:37.051632 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:37.101461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631cfddf-5c3d-41de-8b05-5eb4e3431134 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:37.109090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f490d-e1ac-4957-8bbd-c7cee2387823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:37.109650 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f490d-e1ac-4957-8bbd-c7cee2387823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:37.110103 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f490d-e1ac-4957-8bbd-c7cee2387823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:37.110397 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f490d-e1ac-4957-8bbd-c7cee2387823 admin admin] PATH_INFO: `/v3/users/3954a7b761664f9295983786a007cd56` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:37.112076 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f490d-e1ac-4957-8bbd-c7cee2387823 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3954a7b761664f9295983786a007cd56)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:37.114031 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f490d-e1ac-4957-8bbd-c7cee2387823 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:37.308153 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1f2f490d-e1ac-4957-8bbd-c7cee2387823 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3954a7b761664f9295983786a007cd56'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:37.326205 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1f2f490d-e1ac-4957-8bbd-c7cee2387823 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3954a7b761664f9295983786a007cd56'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:37.397252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1f2f490d-e1ac-4957-8bbd-c7cee2387823 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3954a7b761664f9295983786a007cd56'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:37.398184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1f2f490d-e1ac-4957-8bbd-c7cee2387823 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3954a7b761664f9295983786a007cd56'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:37.400527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1f2f490d-e1ac-4957-8bbd-c7cee2387823 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3954a7b761664f9295983786a007cd56'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:37.454774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1086/2153] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:43:36 2022] DELETE /identity/v3/users/3954a7b761664f9295983786a007cd56 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:37.458658 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d49bf7-a9d4-48b2-89d5-35fb6cc50a06 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:37.458658 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d49bf7-a9d4-48b2-89d5-35fb6cc50a06 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:37.458658 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d49bf7-a9d4-48b2-89d5-35fb6cc50a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:37.471602 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-30d49bf7-a9d4-48b2-89d5-35fb6cc50a06 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:37.472825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-30d49bf7-a9d4-48b2-89d5-35fb6cc50a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:37.482489 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:37.538292 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1068/2154] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:37.546759 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f490d-e1ac-4957-8bbd-c7cee2387823 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:37.548325 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1f2f490d-e1ac-4957-8bbd-c7cee2387823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:37.615849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f490d-e1ac-4957-8bbd-c7cee2387823 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:37.619997 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a3382-12c9-42e2-9863-4e1338234877 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:37.620635 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a3382-12c9-42e2-9863-4e1338234877 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:37.620919 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a3382-12c9-42e2-9863-4e1338234877 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:37.621198 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a3382-12c9-42e2-9863-4e1338234877 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:37.634009 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2a3382-12c9-42e2-9863-4e1338234877 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:37.651048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2a3382-12c9-42e2-9863-4e1338234877 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:37.660921 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1087/2155] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:43:38.216337 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d49bf7-a9d4-48b2-89d5-35fb6cc50a06 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:38.216967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-30d49bf7-a9d4-48b2-89d5-35fb6cc50a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:38.232913 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d49bf7-a9d4-48b2-89d5-35fb6cc50a06 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:38.239168 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323747c-8d2a-43e5-9469-206bef9e96b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:38.240486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323747c-8d2a-43e5-9469-206bef9e96b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:38.240864 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323747c-8d2a-43e5-9469-206bef9e96b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:38.241220 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323747c-8d2a-43e5-9469-206bef9e96b8 admin admin] PATH_INFO: `/v3/projects/56bd474411ba4c59a6fafbdc62313a17` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:38.242582 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323747c-8d2a-43e5-9469-206bef9e96b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56bd474411ba4c59a6fafbdc62313a17)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:38.244827 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323747c-8d2a-43e5-9469-206bef9e96b8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:38.398949 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6323747c-8d2a-43e5-9469-206bef9e96b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56bd474411ba4c59a6fafbdc62313a17', 'request_id': 'req-6323747c-8d2a-43e5-9469-206bef9e96b8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:38.407502 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6323747c-8d2a-43e5-9469-206bef9e96b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56bd474411ba4c59a6fafbdc62313a17', 'request_id': 'req-6323747c-8d2a-43e5-9469-206bef9e96b8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:38.523170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6323747c-8d2a-43e5-9469-206bef9e96b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56bd474411ba4c59a6fafbdc62313a17', 'request_id': 'req-6323747c-8d2a-43e5-9469-206bef9e96b8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:38.538601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6323747c-8d2a-43e5-9469-206bef9e96b8 admin admin] The token cache is being invalidate because project 56bd474411ba4c59a6fafbdc62313a17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:43:38.539205 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6323747c-8d2a-43e5-9469-206bef9e96b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:38.540158 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6323747c-8d2a-43e5-9469-206bef9e96b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:38.542360 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1069/2156] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:43:38 2022] DELETE /identity/v3/projects/56bd474411ba4c59a6fafbdc62313a17 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:38.546588 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a3382-12c9-42e2-9863-4e1338234877 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:38.556804 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a3382-12c9-42e2-9863-4e1338234877 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:38.609317 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:38.633789 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8c9cd-3032-46b7-b5f8-f1e3b5010bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:38.635030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8c9cd-3032-46b7-b5f8-f1e3b5010bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:38.635352 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8c9cd-3032-46b7-b5f8-f1e3b5010bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:38.635803 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8c9cd-3032-46b7-b5f8-f1e3b5010bc7 admin admin] PATH_INFO: `/v3/users/c77a6955f16d4e65bd519b877f1dea3e` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:38.637419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d8c9cd-3032-46b7-b5f8-f1e3b5010bc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c77a6955f16d4e65bd519b877f1dea3e)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:38.639605 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d8c9cd-3032-46b7-b5f8-f1e3b5010bc7 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:38.740277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c14ef-2231-485b-9625-531352907eaa None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:38.740565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c14ef-2231-485b-9625-531352907eaa None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:38.741053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c14ef-2231-485b-9625-531352907eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:38.761282 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-500c14ef-2231-485b-9625-531352907eaa None None] MFA Rules not processed for user `570c2a77b862411083b9444f8292192b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:38.762541 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-500c14ef-2231-485b-9625-531352907eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:38.822089 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1070/2157] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:43:38.935316 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-13d8c9cd-3032-46b7-b5f8-f1e3b5010bc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c77a6955f16d4e65bd519b877f1dea3e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:38.949665 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-13d8c9cd-3032-46b7-b5f8-f1e3b5010bc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c77a6955f16d4e65bd519b877f1dea3e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:39.014108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-13d8c9cd-3032-46b7-b5f8-f1e3b5010bc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c77a6955f16d4e65bd519b877f1dea3e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:39.014722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-13d8c9cd-3032-46b7-b5f8-f1e3b5010bc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c77a6955f16d4e65bd519b877f1dea3e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:39.015598 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-13d8c9cd-3032-46b7-b5f8-f1e3b5010bc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c77a6955f16d4e65bd519b877f1dea3e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:39.040360 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1088/2158] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:43:38 2022] DELETE /identity/v3/users/c77a6955f16d4e65bd519b877f1dea3e => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:39.858642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c14ef-2231-485b-9625-531352907eaa None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:39.864317 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-500c14ef-2231-485b-9625-531352907eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:39.883974 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c14ef-2231-485b-9625-531352907eaa None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:39.964164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:39.985121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f91cd-205f-40c2-96f1-10a00fed3a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:39.985903 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f91cd-205f-40c2-96f1-10a00fed3a6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:39.986250 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f91cd-205f-40c2-96f1-10a00fed3a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:39.988812 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f91cd-205f-40c2-96f1-10a00fed3a6a admin admin] PATH_INFO: `/v3/projects/c629c2fd570f4b44bc6c5015890d8965` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:39.988812 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55f91cd-205f-40c2-96f1-10a00fed3a6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c629c2fd570f4b44bc6c5015890d8965)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:39.991159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55f91cd-205f-40c2-96f1-10a00fed3a6a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:40.152989 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f55f91cd-205f-40c2-96f1-10a00fed3a6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c629c2fd570f4b44bc6c5015890d8965', 'request_id': 'req-f55f91cd-205f-40c2-96f1-10a00fed3a6a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:40.159778 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f55f91cd-205f-40c2-96f1-10a00fed3a6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c629c2fd570f4b44bc6c5015890d8965', 'request_id': 'req-f55f91cd-205f-40c2-96f1-10a00fed3a6a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:40.236396 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f55f91cd-205f-40c2-96f1-10a00fed3a6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c629c2fd570f4b44bc6c5015890d8965', 'request_id': 'req-f55f91cd-205f-40c2-96f1-10a00fed3a6a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:40.249852 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f55f91cd-205f-40c2-96f1-10a00fed3a6a admin admin] The token cache is being invalidate because project c629c2fd570f4b44bc6c5015890d8965 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:43:40.250153 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f55f91cd-205f-40c2-96f1-10a00fed3a6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:40.253047 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f55f91cd-205f-40c2-96f1-10a00fed3a6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:40.253047 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1071/2159] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:43:39 2022] DELETE /identity/v3/projects/c629c2fd570f4b44bc6c5015890d8965 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:40.257724 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e1edc-ef68-41a5-ae86-d50c82539d2e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:40.258556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e1edc-ef68-41a5-ae86-d50c82539d2e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:40.258836 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e1edc-ef68-41a5-ae86-d50c82539d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:40.281245 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-874e1edc-ef68-41a5-ae86-d50c82539d2e None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:40.282554 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-874e1edc-ef68-41a5-ae86-d50c82539d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:40.342978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1089/2160] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:43:40.352456 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78d7c6-39e2-4046-a8a5-4568a90115d3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:40.352749 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78d7c6-39e2-4046-a8a5-4568a90115d3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:40.353055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78d7c6-39e2-4046-a8a5-4568a90115d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:40.373841 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-fa78d7c6-39e2-4046-a8a5-4568a90115d3 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:40.375734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fa78d7c6-39e2-4046-a8a5-4568a90115d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:40.383735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1072/2161] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:40.388117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874e1edc-ef68-41a5-ae86-d50c82539d2e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:40.397806 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874e1edc-ef68-41a5-ae86-d50c82539d2e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:40.402372 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188fb037-634a-4f5a-a5c5-977596dfb95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:40.402968 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fb037-634a-4f5a-a5c5-977596dfb95a admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:40.403278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fb037-634a-4f5a-a5c5-977596dfb95a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:40.403555 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fb037-634a-4f5a-a5c5-977596dfb95a admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:40.404222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188fb037-634a-4f5a-a5c5-977596dfb95a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:43:40.404528 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188fb037-634a-4f5a-a5c5-977596dfb95a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:40.406121 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188fb037-634a-4f5a-a5c5-977596dfb95a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:40.417514 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1090/2162] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:43:40.422637 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78d7c6-39e2-4046-a8a5-4568a90115d3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:40.431920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78d7c6-39e2-4046-a8a5-4568a90115d3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:40.437356 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d63cc-eb66-4091-8951-9501043e35ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:40.437978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d63cc-eb66-4091-8951-9501043e35ad admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:40.438260 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d63cc-eb66-4091-8951-9501043e35ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:40.438685 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d63cc-eb66-4091-8951-9501043e35ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:40.439153 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d63cc-eb66-4091-8951-9501043e35ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:40.441188 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d63cc-eb66-4091-8951-9501043e35ad admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:40.556065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1073/2163] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:43:40 2022] POST /identity/v3/projects => generated 379 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:43:40.557207 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188fb037-634a-4f5a-a5c5-977596dfb95a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:40.569269 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188fb037-634a-4f5a-a5c5-977596dfb95a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:40.626291 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:40.640993 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a37c5-1214-4ad6-b5a7-93905be3d27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:40.641858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a37c5-1214-4ad6-b5a7-93905be3d27d admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:40.642135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a37c5-1214-4ad6-b5a7-93905be3d27d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:40.644036 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a37c5-1214-4ad6-b5a7-93905be3d27d admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:40.644036 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a37c5-1214-4ad6-b5a7-93905be3d27d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:40.645047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a37c5-1214-4ad6-b5a7-93905be3d27d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:40.784301 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1091/2164] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:43:40 2022] POST /identity/v3/users => generated 290 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:43:40.787511 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d63cc-eb66-4091-8951-9501043e35ad admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:40.799065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d63cc-eb66-4091-8951-9501043e35ad admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:40.804206 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0010e64-7114-452f-85c1-3b92cb92d12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:40.804809 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0010e64-7114-452f-85c1-3b92cb92d12a admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:40.805090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0010e64-7114-452f-85c1-3b92cb92d12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:40.805434 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0010e64-7114-452f-85c1-3b92cb92d12a admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:40.806107 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0010e64-7114-452f-85c1-3b92cb92d12a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:40.807732 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0010e64-7114-452f-85c1-3b92cb92d12a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:40.819626 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1074/2165] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:43:40.823775 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a37c5-1214-4ad6-b5a7-93905be3d27d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:40.835405 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a37c5-1214-4ad6-b5a7-93905be3d27d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:40.840898 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f6248-d1f7-4185-8944-be2d425e313f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:40.841485 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f6248-d1f7-4185-8944-be2d425e313f admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:40.841762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f6248-d1f7-4185-8944-be2d425e313f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:40.842034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f6248-d1f7-4185-8944-be2d425e313f admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:40.843486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f6248-d1f7-4185-8944-be2d425e313f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:40.845792 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f6248-d1f7-4185-8944-be2d425e313f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:40.856764 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1092/2166] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:43:40.862632 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0010e64-7114-452f-85c1-3b92cb92d12a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:40.874094 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0010e64-7114-452f-85c1-3b92cb92d12a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:40.878739 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35548210-de26-4cb7-83b7-96513a9f7e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:40.879519 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35548210-de26-4cb7-83b7-96513a9f7e01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:40.879912 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35548210-de26-4cb7-83b7-96513a9f7e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:40.880184 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35548210-de26-4cb7-83b7-96513a9f7e01 admin admin] PATH_INFO: `/v3/projects/e1f5f774917a4e7780479ece1c69eaa6/users/eda32f208f104128a711b64ab45822fe/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:40.895926 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35548210-de26-4cb7-83b7-96513a9f7e01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1f5f774917a4e7780479ece1c69eaa6, user_id=eda32f208f104128a711b64ab45822fe, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:40.898552 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35548210-de26-4cb7-83b7-96513a9f7e01 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:40.998099 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1075/2167] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:43:40 2022] PUT /identity/v3/projects/e1f5f774917a4e7780479ece1c69eaa6/users/eda32f208f104128a711b64ab45822fe/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:41.008702 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60790fa7-b2ca-4153-872a-543188b35dfb None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:41.008998 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60790fa7-b2ca-4153-872a-543188b35dfb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:41.009490 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60790fa7-b2ca-4153-872a-543188b35dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:41.043216 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-60790fa7-b2ca-4153-872a-543188b35dfb None None] MFA Rules not processed for user `eda32f208f104128a711b64ab45822fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:41.046827 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-60790fa7-b2ca-4153-872a-543188b35dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:41.123614 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1093/2168] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:41.128209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35548210-de26-4cb7-83b7-96513a9f7e01 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:41.205221 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35548210-de26-4cb7-83b7-96513a9f7e01 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:41.210201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa227db-ddac-402c-8368-84fb5396b3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:41.210842 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa227db-ddac-402c-8368-84fb5396b3ea admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:41.218606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa227db-ddac-402c-8368-84fb5396b3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:41.218606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa227db-ddac-402c-8368-84fb5396b3ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:41.218606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa227db-ddac-402c-8368-84fb5396b3ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:41.218606 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa227db-ddac-402c-8368-84fb5396b3ea admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:41.231640 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:41.294833 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1076/2169] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:43:41 2022] POST /identity/v3/projects => generated 381 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:43:41.294983 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60790fa7-b2ca-4153-872a-543188b35dfb None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:41.305627 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60790fa7-b2ca-4153-872a-543188b35dfb None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:41.368179 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73220870-4e60-43fa-b925-819c4fc20313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:41.368830 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73220870-4e60-43fa-b925-819c4fc20313 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:41.369115 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73220870-4e60-43fa-b925-819c4fc20313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:41.369392 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73220870-4e60-43fa-b925-819c4fc20313 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:41.370036 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73220870-4e60-43fa-b925-819c4fc20313 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:41.371989 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73220870-4e60-43fa-b925-819c4fc20313 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:41.477011 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1094/2170] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:43:41 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:43:41.480939 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa227db-ddac-402c-8368-84fb5396b3ea admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:41.492328 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa227db-ddac-402c-8368-84fb5396b3ea admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:41.497229 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f1fd2-d485-4319-8a0a-79dfc2a1ef95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:41.497864 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f1fd2-d485-4319-8a0a-79dfc2a1ef95 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:41.498148 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f1fd2-d485-4319-8a0a-79dfc2a1ef95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:41.498427 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f1fd2-d485-4319-8a0a-79dfc2a1ef95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:41.499651 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6f1fd2-d485-4319-8a0a-79dfc2a1ef95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:41.501344 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6f1fd2-d485-4319-8a0a-79dfc2a1ef95 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:41.513251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1077/2171] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:43:41.521213 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73220870-4e60-43fa-b925-819c4fc20313 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:41.531949 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73220870-4e60-43fa-b925-819c4fc20313 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:41.536688 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f38771-bd15-4ab6-8f37-ecdb67917e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:41.537344 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f38771-bd15-4ab6-8f37-ecdb67917e64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:41.537624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f38771-bd15-4ab6-8f37-ecdb67917e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:41.537919 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f38771-bd15-4ab6-8f37-ecdb67917e64 admin admin] PATH_INFO: `/v3/projects/f5f56bb11379409fb9bbedf5579f1082/users/97f6c67204404c8898eaf8b687e3d040/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:41.553660 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f38771-bd15-4ab6-8f37-ecdb67917e64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5f56bb11379409fb9bbedf5579f1082, user_id=97f6c67204404c8898eaf8b687e3d040, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:41.556360 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f38771-bd15-4ab6-8f37-ecdb67917e64 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:41.671798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1095/2172] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:43:41 2022] PUT /identity/v3/projects/f5f56bb11379409fb9bbedf5579f1082/users/97f6c67204404c8898eaf8b687e3d040/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:41.675875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f1fd2-d485-4319-8a0a-79dfc2a1ef95 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:41.686090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f1fd2-d485-4319-8a0a-79dfc2a1ef95 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:41.779717 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cf53a-617d-433c-a737-26de5ecef7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:41.780802 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cf53a-617d-433c-a737-26de5ecef7c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:41.781838 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cf53a-617d-433c-a737-26de5ecef7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:41.782141 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cf53a-617d-433c-a737-26de5ecef7c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:41.782794 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785cf53a-617d-433c-a737-26de5ecef7c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:41.784636 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785cf53a-617d-433c-a737-26de5ecef7c9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:41.815239 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1078/2173] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:43:41.819324 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f38771-bd15-4ab6-8f37-ecdb67917e64 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:41.827631 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f38771-bd15-4ab6-8f37-ecdb67917e64 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:41.832716 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aacbf99-3d1f-491c-b8da-20e0d155d9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:41.833316 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aacbf99-3d1f-491c-b8da-20e0d155d9ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:41.833603 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aacbf99-3d1f-491c-b8da-20e0d155d9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:41.833905 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aacbf99-3d1f-491c-b8da-20e0d155d9ca admin admin] PATH_INFO: `/v3/projects/f5f56bb11379409fb9bbedf5579f1082/users/97f6c67204404c8898eaf8b687e3d040/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:41.836896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aacbf99-3d1f-491c-b8da-20e0d155d9ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5f56bb11379409fb9bbedf5579f1082, user_id=97f6c67204404c8898eaf8b687e3d040, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:41.839399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aacbf99-3d1f-491c-b8da-20e0d155d9ca admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:41.853146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1096/2174] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:43:41 2022] PUT /identity/v3/projects/f5f56bb11379409fb9bbedf5579f1082/users/97f6c67204404c8898eaf8b687e3d040/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:43:41.865195 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a93296-ba22-4947-8031-cd9d99bce6e3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:41.865752 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a93296-ba22-4947-8031-cd9d99bce6e3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:41.865851 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a93296-ba22-4947-8031-cd9d99bce6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:41.904214 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d1a93296-ba22-4947-8031-cd9d99bce6e3 None None] MFA Rules not processed for user `97f6c67204404c8898eaf8b687e3d040`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:41.905619 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d1a93296-ba22-4947-8031-cd9d99bce6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:41.987985 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1079/2175] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:41 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:41.998851 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aacbf99-3d1f-491c-b8da-20e0d155d9ca admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:42.002363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4aacbf99-3d1f-491c-b8da-20e0d155d9ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:42.037070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:42.102864 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aacbf99-3d1f-491c-b8da-20e0d155d9ca admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:42.107469 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672fc85-dd08-45e0-a187-0d6e178808db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:42.108233 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672fc85-dd08-45e0-a187-0d6e178808db service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:42.109769 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672fc85-dd08-45e0-a187-0d6e178808db service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:42.110089 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672fc85-dd08-45e0-a187-0d6e178808db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:42.122247 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c672fc85-dd08-45e0-a187-0d6e178808db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:42.125480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c672fc85-dd08-45e0-a187-0d6e178808db service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:42.142297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1097/2176] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:41 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:43:42.797253 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a93296-ba22-4947-8031-cd9d99bce6e3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:42.884140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a93296-ba22-4947-8031-cd9d99bce6e3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:42.889077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c76d9-609a-4d88-a08c-d702740e1ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:42.890617 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c76d9-609a-4d88-a08c-d702740e1ecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:42.891134 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c76d9-609a-4d88-a08c-d702740e1ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:42.891650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c76d9-609a-4d88-a08c-d702740e1ecb admin admin] PATH_INFO: `/v3/users/eda32f208f104128a711b64ab45822fe` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:42.893237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c76d9-609a-4d88-a08c-d702740e1ecb admin admin] RBAC: Authorizing `identity:delete_user(user_id=eda32f208f104128a711b64ab45822fe)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:42.895235 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c76d9-609a-4d88-a08c-d702740e1ecb admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:42.901165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:43.112187 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-246c76d9-609a-4d88-a08c-d702740e1ecb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eda32f208f104128a711b64ab45822fe'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:43.162204 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-246c76d9-609a-4d88-a08c-d702740e1ecb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eda32f208f104128a711b64ab45822fe'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:43.162877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-246c76d9-609a-4d88-a08c-d702740e1ecb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eda32f208f104128a711b64ab45822fe'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:43.163517 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-246c76d9-609a-4d88-a08c-d702740e1ecb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eda32f208f104128a711b64ab45822fe'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:43.178096 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-246c76d9-609a-4d88-a08c-d702740e1ecb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eda32f208f104128a711b64ab45822fe'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:43.196478 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1080/2177] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:43:42 2022] DELETE /identity/v3/users/eda32f208f104128a711b64ab45822fe => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:43.201088 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81d48e-950b-4226-b486-d70accbac87c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:43.201577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81d48e-950b-4226-b486-d70accbac87c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:43.201683 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81d48e-950b-4226-b486-d70accbac87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:43.222187 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-1d81d48e-950b-4226-b486-d70accbac87c None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:43.223506 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1d81d48e-950b-4226-b486-d70accbac87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:43.302135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1098/2178] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:43.312323 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c76d9-609a-4d88-a08c-d702740e1ecb admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:43.313851 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-246c76d9-609a-4d88-a08c-d702740e1ecb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:43.380603 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c76d9-609a-4d88-a08c-d702740e1ecb admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:43.384320 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677fea98-e955-4fe5-af19-0d25812eb81b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:43.384930 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677fea98-e955-4fe5-af19-0d25812eb81b service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:43.385210 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677fea98-e955-4fe5-af19-0d25812eb81b service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:43.385481 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677fea98-e955-4fe5-af19-0d25812eb81b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:43.397170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677fea98-e955-4fe5-af19-0d25812eb81b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:43.398643 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677fea98-e955-4fe5-af19-0d25812eb81b service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:43.409921 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1081/2179] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:43:43.827858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81d48e-950b-4226-b486-d70accbac87c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:43.829304 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1d81d48e-950b-4226-b486-d70accbac87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:43.841590 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81d48e-950b-4226-b486-d70accbac87c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:43.846332 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911b564-63ac-4030-811c-ddd95280c69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:43.848539 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911b564-63ac-4030-811c-ddd95280c69a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:43.848539 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911b564-63ac-4030-811c-ddd95280c69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:43.848539 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911b564-63ac-4030-811c-ddd95280c69a admin admin] PATH_INFO: `/v3/projects/e1f5f774917a4e7780479ece1c69eaa6` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:43.856564 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911b564-63ac-4030-811c-ddd95280c69a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1f5f774917a4e7780479ece1c69eaa6)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:43.858621 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911b564-63ac-4030-811c-ddd95280c69a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:43.968355 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1911b564-63ac-4030-811c-ddd95280c69a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1f5f774917a4e7780479ece1c69eaa6', 'request_id': 'req-1911b564-63ac-4030-811c-ddd95280c69a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:43.973924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1911b564-63ac-4030-811c-ddd95280c69a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1f5f774917a4e7780479ece1c69eaa6', 'request_id': 'req-1911b564-63ac-4030-811c-ddd95280c69a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:44.041976 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1911b564-63ac-4030-811c-ddd95280c69a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1f5f774917a4e7780479ece1c69eaa6', 'request_id': 'req-1911b564-63ac-4030-811c-ddd95280c69a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:44.054257 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1911b564-63ac-4030-811c-ddd95280c69a admin admin] The token cache is being invalidate because project e1f5f774917a4e7780479ece1c69eaa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:43:44.054609 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1911b564-63ac-4030-811c-ddd95280c69a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:44.055284 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1911b564-63ac-4030-811c-ddd95280c69a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:44.056881 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1099/2180] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:43:43 2022] DELETE /identity/v3/projects/e1f5f774917a4e7780479ece1c69eaa6 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:44.060938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677fea98-e955-4fe5-af19-0d25812eb81b service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:44.071412 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677fea98-e955-4fe5-af19-0d25812eb81b service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:44.142135 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee0fee-b760-4ce1-a1c9-c8fefb09a861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:44.142703 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee0fee-b760-4ce1-a1c9-c8fefb09a861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:44.142986 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee0fee-b760-4ce1-a1c9-c8fefb09a861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:44.143314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee0fee-b760-4ce1-a1c9-c8fefb09a861 admin admin] PATH_INFO: `/v3/users/97f6c67204404c8898eaf8b687e3d040` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:44.144461 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee0fee-b760-4ce1-a1c9-c8fefb09a861 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97f6c67204404c8898eaf8b687e3d040)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:44.146381 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee0fee-b760-4ce1-a1c9-c8fefb09a861 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:44.340306 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:44.374800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b9ee0fee-b760-4ce1-a1c9-c8fefb09a861 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97f6c67204404c8898eaf8b687e3d040'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:44.482495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b9ee0fee-b760-4ce1-a1c9-c8fefb09a861 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97f6c67204404c8898eaf8b687e3d040'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:44.483146 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b9ee0fee-b760-4ce1-a1c9-c8fefb09a861 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97f6c67204404c8898eaf8b687e3d040'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:44.484590 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b9ee0fee-b760-4ce1-a1c9-c8fefb09a861 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97f6c67204404c8898eaf8b687e3d040'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:44.498333 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b9ee0fee-b760-4ce1-a1c9-c8fefb09a861 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97f6c67204404c8898eaf8b687e3d040'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:44.558734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1082/2181] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:43:44 2022] DELETE /identity/v3/users/97f6c67204404c8898eaf8b687e3d040 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:45.410128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911b564-63ac-4030-811c-ddd95280c69a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:45.414187 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1911b564-63ac-4030-811c-ddd95280c69a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:45.457468 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911b564-63ac-4030-811c-ddd95280c69a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:45.519159 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:45.546743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5bc844-fa02-467d-bf2e-4ca85aed41d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:45.547500 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5bc844-fa02-467d-bf2e-4ca85aed41d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:45.547700 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5bc844-fa02-467d-bf2e-4ca85aed41d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:45.548035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5bc844-fa02-467d-bf2e-4ca85aed41d4 admin admin] PATH_INFO: `/v3/projects/f5f56bb11379409fb9bbedf5579f1082` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:45.549191 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5bc844-fa02-467d-bf2e-4ca85aed41d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5f56bb11379409fb9bbedf5579f1082)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:45.551253 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5bc844-fa02-467d-bf2e-4ca85aed41d4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:45.654786 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0b5bc844-fa02-467d-bf2e-4ca85aed41d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5f56bb11379409fb9bbedf5579f1082', 'request_id': 'req-0b5bc844-fa02-467d-bf2e-4ca85aed41d4'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:45.661398 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0b5bc844-fa02-467d-bf2e-4ca85aed41d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5f56bb11379409fb9bbedf5579f1082', 'request_id': 'req-0b5bc844-fa02-467d-bf2e-4ca85aed41d4'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:45.740135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0b5bc844-fa02-467d-bf2e-4ca85aed41d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5f56bb11379409fb9bbedf5579f1082', 'request_id': 'req-0b5bc844-fa02-467d-bf2e-4ca85aed41d4'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:45.752315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0b5bc844-fa02-467d-bf2e-4ca85aed41d4 admin admin] The token cache is being invalidate because project f5f56bb11379409fb9bbedf5579f1082 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:43:45.752604 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0b5bc844-fa02-467d-bf2e-4ca85aed41d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:45.753304 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0b5bc844-fa02-467d-bf2e-4ca85aed41d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:45.754834 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1100/2182] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:43:45 2022] DELETE /identity/v3/projects/f5f56bb11379409fb9bbedf5579f1082 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:45.759840 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a8d1f-0572-4d1f-9afd-174bb4428cce None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:45.760301 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a8d1f-0572-4d1f-9afd-174bb4428cce None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:45.763011 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a8d1f-0572-4d1f-9afd-174bb4428cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:45.781780 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e16a8d1f-0572-4d1f-9afd-174bb4428cce None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:45.782988 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e16a8d1f-0572-4d1f-9afd-174bb4428cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:45.842065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1083/2183] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:45.850914 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1e355-e83b-4a6e-ad7d-218d37ebfb29 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:45.851521 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1e355-e83b-4a6e-ad7d-218d37ebfb29 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:45.851820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1e355-e83b-4a6e-ad7d-218d37ebfb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:45.872264 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-4fa1e355-e83b-4a6e-ad7d-218d37ebfb29 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:45.873590 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4fa1e355-e83b-4a6e-ad7d-218d37ebfb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:45.884355 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1101/2184] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:45.888539 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a8d1f-0572-4d1f-9afd-174bb4428cce None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:45.899501 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a8d1f-0572-4d1f-9afd-174bb4428cce None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:45.904520 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a86546-5feb-4d18-afbb-e1e152002b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:45.905282 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a86546-5feb-4d18-afbb-e1e152002b8e admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:45.905749 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a86546-5feb-4d18-afbb-e1e152002b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:45.906303 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a86546-5feb-4d18-afbb-e1e152002b8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:45.907143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a86546-5feb-4d18-afbb-e1e152002b8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:43:45.907551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a86546-5feb-4d18-afbb-e1e152002b8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:45.909228 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a86546-5feb-4d18-afbb-e1e152002b8e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:45.921740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1084/2185] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:43:45.925861 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1e355-e83b-4a6e-ad7d-218d37ebfb29 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:45.935090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1e355-e83b-4a6e-ad7d-218d37ebfb29 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:45.940417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e3e86-3a28-4ded-9f6e-1a4581f080a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:45.941087 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e3e86-3a28-4ded-9f6e-1a4581f080a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:45.941454 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e3e86-3a28-4ded-9f6e-1a4581f080a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:45.941787 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e3e86-3a28-4ded-9f6e-1a4581f080a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:45.942565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e3e86-3a28-4ded-9f6e-1a4581f080a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:45.944557 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e3e86-3a28-4ded-9f6e-1a4581f080a8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:46.053643 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1102/2186] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:43:45 2022] POST /identity/v3/projects => generated 371 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:43:46.054862 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a86546-5feb-4d18-afbb-e1e152002b8e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:46.082299 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a86546-5feb-4d18-afbb-e1e152002b8e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:46.142109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c8d49-247f-484b-8b3c-5295283d6cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:46.142750 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c8d49-247f-484b-8b3c-5295283d6cf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:46.143017 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c8d49-247f-484b-8b3c-5295283d6cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:46.143317 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c8d49-247f-484b-8b3c-5295283d6cf0 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:46.143943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c8d49-247f-484b-8b3c-5295283d6cf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:46.148892 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c8d49-247f-484b-8b3c-5295283d6cf0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:46.221622 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1085/2187] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:43:46 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:43:46.226048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e3e86-3a28-4ded-9f6e-1a4581f080a8 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:46.237107 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e3e86-3a28-4ded-9f6e-1a4581f080a8 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:46.242857 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce1142-2ff8-46c1-b185-0a145b2eecb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:46.243467 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce1142-2ff8-46c1-b185-0a145b2eecb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:46.243747 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce1142-2ff8-46c1-b185-0a145b2eecb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:46.244028 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce1142-2ff8-46c1-b185-0a145b2eecb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:46.244661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce1142-2ff8-46c1-b185-0a145b2eecb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:46.246254 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce1142-2ff8-46c1-b185-0a145b2eecb8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:46.257676 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1103/2188] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:43:46.264187 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c8d49-247f-484b-8b3c-5295283d6cf0 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:46.288182 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c8d49-247f-484b-8b3c-5295283d6cf0 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:46.293906 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c1d09-e3a7-4293-a847-bfd34e9a5eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:46.294646 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c1d09-e3a7-4293-a847-bfd34e9a5eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:46.294985 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c1d09-e3a7-4293-a847-bfd34e9a5eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:46.296228 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c1d09-e3a7-4293-a847-bfd34e9a5eb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:46.297242 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c1d09-e3a7-4293-a847-bfd34e9a5eb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:46.299644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c1d09-e3a7-4293-a847-bfd34e9a5eb5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:46.310839 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1086/2189] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:43:46.320512 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce1142-2ff8-46c1-b185-0a145b2eecb8 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:46.332974 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce1142-2ff8-46c1-b185-0a145b2eecb8 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:46.337936 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400900c8-9e7f-40ee-a0c8-7e8ab9f4480c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:46.338515 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400900c8-9e7f-40ee-a0c8-7e8ab9f4480c admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:46.338802 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400900c8-9e7f-40ee-a0c8-7e8ab9f4480c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:46.339074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400900c8-9e7f-40ee-a0c8-7e8ab9f4480c admin admin] PATH_INFO: `/v3/projects/3ab06c6d54cc42e9860f4e67f5981889/users/61174d2087e14c88b89e6b8af78121a7/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:46.355069 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400900c8-9e7f-40ee-a0c8-7e8ab9f4480c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ab06c6d54cc42e9860f4e67f5981889, user_id=61174d2087e14c88b89e6b8af78121a7, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:46.357760 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400900c8-9e7f-40ee-a0c8-7e8ab9f4480c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:46.449025 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1104/2190] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:43:46 2022] PUT /identity/v3/projects/3ab06c6d54cc42e9860f4e67f5981889/users/61174d2087e14c88b89e6b8af78121a7/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:46.460361 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcbae1-4724-4108-a044-064a6014115f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:46.460651 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcbae1-4724-4108-a044-064a6014115f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:46.460929 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcbae1-4724-4108-a044-064a6014115f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:46.486581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:46.489681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-01bcbae1-4724-4108-a044-064a6014115f None None] MFA Rules not processed for user `61174d2087e14c88b89e6b8af78121a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:46.491014 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-01bcbae1-4724-4108-a044-064a6014115f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:46.567037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1087/2191] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:46.578902 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400900c8-9e7f-40ee-a0c8-7e8ab9f4480c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:46.582909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-400900c8-9e7f-40ee-a0c8-7e8ab9f4480c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:46.643852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400900c8-9e7f-40ee-a0c8-7e8ab9f4480c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:46.648361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79a2ed6-7fe5-4520-bccc-a79e925ca279 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:46.648361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a2ed6-7fe5-4520-bccc-a79e925ca279 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:46.648763 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a2ed6-7fe5-4520-bccc-a79e925ca279 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:46.649038 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a2ed6-7fe5-4520-bccc-a79e925ca279 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:46.658450 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79a2ed6-7fe5-4520-bccc-a79e925ca279 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:46.659930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79a2ed6-7fe5-4520-bccc-a79e925ca279 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:46.669167 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1105/2192] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:43:46.698527 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bcbae1-4724-4108-a044-064a6014115f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:46.766232 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bcbae1-4724-4108-a044-064a6014115f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:46.773414 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea6bca-54e3-4baa-adea-1557572e7b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:46.774076 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea6bca-54e3-4baa-adea-1557572e7b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:46.781107 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea6bca-54e3-4baa-adea-1557572e7b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:46.781107 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea6bca-54e3-4baa-adea-1557572e7b35 admin admin] PATH_INFO: `/v3/users/61174d2087e14c88b89e6b8af78121a7` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:46.781107 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea6bca-54e3-4baa-adea-1557572e7b35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61174d2087e14c88b89e6b8af78121a7)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:46.781107 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea6bca-54e3-4baa-adea-1557572e7b35 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:47.042890 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-adea6bca-54e3-4baa-adea-1557572e7b35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61174d2087e14c88b89e6b8af78121a7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:47.121981 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-adea6bca-54e3-4baa-adea-1557572e7b35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61174d2087e14c88b89e6b8af78121a7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:47.122625 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-adea6bca-54e3-4baa-adea-1557572e7b35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61174d2087e14c88b89e6b8af78121a7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:47.123470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-adea6bca-54e3-4baa-adea-1557572e7b35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61174d2087e14c88b89e6b8af78121a7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:47.133348 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-adea6bca-54e3-4baa-adea-1557572e7b35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61174d2087e14c88b89e6b8af78121a7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:47.158017 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1088/2193] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:43:46 2022] DELETE /identity/v3/users/61174d2087e14c88b89e6b8af78121a7 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:43:47.163026 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032e1a9-0475-4362-9ac0-06cd3b5cae54 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:47.163300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032e1a9-0475-4362-9ac0-06cd3b5cae54 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:47.163563 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032e1a9-0475-4362-9ac0-06cd3b5cae54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:47.172456 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:47.183273 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3032e1a9-0475-4362-9ac0-06cd3b5cae54 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:47.184566 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3032e1a9-0475-4362-9ac0-06cd3b5cae54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:47.251016 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1106/2194] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:47.261281 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea6bca-54e3-4baa-adea-1557572e7b35 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:47.262811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-adea6bca-54e3-4baa-adea-1557572e7b35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:47.299721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:47.330410 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea6bca-54e3-4baa-adea-1557572e7b35 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:47.333986 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceafbbbf-ab73-4d93-b856-391554cd4efc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:47.334587 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceafbbbf-ab73-4d93-b856-391554cd4efc service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:47.334863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceafbbbf-ab73-4d93-b856-391554cd4efc service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:47.335131 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceafbbbf-ab73-4d93-b856-391554cd4efc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:47.345251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceafbbbf-ab73-4d93-b856-391554cd4efc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:47.346687 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceafbbbf-ab73-4d93-b856-391554cd4efc service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:47.356007 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1089/2195] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:43:47.782047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032e1a9-0475-4362-9ac0-06cd3b5cae54 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:47.783661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3032e1a9-0475-4362-9ac0-06cd3b5cae54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:47.803244 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032e1a9-0475-4362-9ac0-06cd3b5cae54 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:47.815741 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21892f8c-d9b5-40c0-b4b7-bbe087061983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:47.817790 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21892f8c-d9b5-40c0-b4b7-bbe087061983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:47.818133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21892f8c-d9b5-40c0-b4b7-bbe087061983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:47.818427 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21892f8c-d9b5-40c0-b4b7-bbe087061983 admin admin] PATH_INFO: `/v3/projects/3ab06c6d54cc42e9860f4e67f5981889` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:47.822286 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21892f8c-d9b5-40c0-b4b7-bbe087061983 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ab06c6d54cc42e9860f4e67f5981889)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:47.824496 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21892f8c-d9b5-40c0-b4b7-bbe087061983 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:47.920161 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceafbbbf-ab73-4d93-b856-391554cd4efc service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:47.928581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ceafbbbf-ab73-4d93-b856-391554cd4efc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:47.937261 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceafbbbf-ab73-4d93-b856-391554cd4efc service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:47.943324 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-21892f8c-d9b5-40c0-b4b7-bbe087061983 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ab06c6d54cc42e9860f4e67f5981889', 'request_id': 'req-21892f8c-d9b5-40c0-b4b7-bbe087061983'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:47.953195 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-21892f8c-d9b5-40c0-b4b7-bbe087061983 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ab06c6d54cc42e9860f4e67f5981889', 'request_id': 'req-21892f8c-d9b5-40c0-b4b7-bbe087061983'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:47.998356 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44316740-3bf5-453a-ac7e-070bf1c307ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:47.999717 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44316740-3bf5-453a-ac7e-070bf1c307ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:48.000024 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44316740-3bf5-453a-ac7e-070bf1c307ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:48.000334 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44316740-3bf5-453a-ac7e-070bf1c307ef admin admin] PATH_INFO: `/v3/users/07056b0bd25047e189356dc54a29dc69` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:48.001540 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44316740-3bf5-453a-ac7e-070bf1c307ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=07056b0bd25047e189356dc54a29dc69)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:48.003507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44316740-3bf5-453a-ac7e-070bf1c307ef admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:48.054002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-21892f8c-d9b5-40c0-b4b7-bbe087061983 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ab06c6d54cc42e9860f4e67f5981889', 'request_id': 'req-21892f8c-d9b5-40c0-b4b7-bbe087061983'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:48.066833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-21892f8c-d9b5-40c0-b4b7-bbe087061983 admin admin] The token cache is being invalidate because project 3ab06c6d54cc42e9860f4e67f5981889 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:43:48.067131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-21892f8c-d9b5-40c0-b4b7-bbe087061983 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:48.067818 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-21892f8c-d9b5-40c0-b4b7-bbe087061983 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:48.069338 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1107/2196] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:43:47 2022] DELETE /identity/v3/projects/3ab06c6d54cc42e9860f4e67f5981889 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:48.074256 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdf5db-eaf9-4828-a8c0-11555da0d5a4 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:48.074544 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdf5db-eaf9-4828-a8c0-11555da0d5a4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:48.074820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdf5db-eaf9-4828-a8c0-11555da0d5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:48.096847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-54bdf5db-eaf9-4828-a8c0-11555da0d5a4 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:48.098199 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-54bdf5db-eaf9-4828-a8c0-11555da0d5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:48.110649 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1108/2197] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:43:48.119375 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8ab38-312d-4937-a070-738adbab89cb None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:48.119670 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8ab38-312d-4937-a070-738adbab89cb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:48.119972 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8ab38-312d-4937-a070-738adbab89cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:48.141328 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6fa8ab38-312d-4937-a070-738adbab89cb None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:48.142711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6fa8ab38-312d-4937-a070-738adbab89cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:48.159223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1109/2198] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:48.166496 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8ab38-312d-4937-a070-738adbab89cb None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:48.170187 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8ab38-312d-4937-a070-738adbab89cb None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:48.177007 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec67f4-8e98-4a07-83bc-ffb84d71147e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:48.177827 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec67f4-8e98-4a07-83bc-ffb84d71147e admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:48.178289 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec67f4-8e98-4a07-83bc-ffb84d71147e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:48.178786 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec67f4-8e98-4a07-83bc-ffb84d71147e admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:48.180389 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec67f4-8e98-4a07-83bc-ffb84d71147e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:43:48.180845 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec67f4-8e98-4a07-83bc-ffb84d71147e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:48.182798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec67f4-8e98-4a07-83bc-ffb84d71147e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:48.215124 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1110/2199] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:43:48.216197 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec67f4-8e98-4a07-83bc-ffb84d71147e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:48.241534 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec67f4-8e98-4a07-83bc-ffb84d71147e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:48.243551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-44316740-3bf5-453a-ac7e-070bf1c307ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07056b0bd25047e189356dc54a29dc69'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:48.248116 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8bffb-8ce5-4cef-b6b9-0aaec684f69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:48.248723 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8bffb-8ce5-4cef-b6b9-0aaec684f69f admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:48.249930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8bffb-8ce5-4cef-b6b9-0aaec684f69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:48.250766 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8bffb-8ce5-4cef-b6b9-0aaec684f69f admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:48.251668 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d8bffb-8ce5-4cef-b6b9-0aaec684f69f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:48.253826 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d8bffb-8ce5-4cef-b6b9-0aaec684f69f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:48.346162 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-44316740-3bf5-453a-ac7e-070bf1c307ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07056b0bd25047e189356dc54a29dc69'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:48.346785 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-44316740-3bf5-453a-ac7e-070bf1c307ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07056b0bd25047e189356dc54a29dc69'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:48.347428 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-44316740-3bf5-453a-ac7e-070bf1c307ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07056b0bd25047e189356dc54a29dc69'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:48.356346 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-44316740-3bf5-453a-ac7e-070bf1c307ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07056b0bd25047e189356dc54a29dc69'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:48.403477 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1111/2200] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:43:48 2022] POST /identity/v3/projects => generated 375 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:43:48.408231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8bffb-8ce5-4cef-b6b9-0aaec684f69f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:48.411201 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-02d8bffb-8ce5-4cef-b6b9-0aaec684f69f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:48.451308 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1090/2201] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:43:47 2022] DELETE /identity/v3/users/07056b0bd25047e189356dc54a29dc69 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:48.459468 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026595b-a23d-4a95-9b33-cec584fb8d53 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:48.459760 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026595b-a23d-4a95-9b33-cec584fb8d53 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:48.460026 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026595b-a23d-4a95-9b33-cec584fb8d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:48.545590 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-2026595b-a23d-4a95-9b33-cec584fb8d53 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:48.549555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2026595b-a23d-4a95-9b33-cec584fb8d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:48.593868 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:48.672063 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:48.675466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8bffb-8ce5-4cef-b6b9-0aaec684f69f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:48.727905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1091/2202] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:48.783049 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2026595b-a23d-4a95-9b33-cec584fb8d53 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:48.784886 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2026595b-a23d-4a95-9b33-cec584fb8d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:48.805776 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5150d39-9db3-486a-9c96-07d9ee2df160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:48.807355 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5150d39-9db3-486a-9c96-07d9ee2df160 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:48.808701 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5150d39-9db3-486a-9c96-07d9ee2df160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:48.809212 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5150d39-9db3-486a-9c96-07d9ee2df160 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:48.810070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5150d39-9db3-486a-9c96-07d9ee2df160 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:48.814055 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5150d39-9db3-486a-9c96-07d9ee2df160 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:48.857044 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2026595b-a23d-4a95-9b33-cec584fb8d53 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:48.863468 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28316f48-12e7-4420-a787-a885f55fe844 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:48.864995 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28316f48-12e7-4420-a787-a885f55fe844 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:48.864995 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28316f48-12e7-4420-a787-a885f55fe844 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:48.864995 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28316f48-12e7-4420-a787-a885f55fe844 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:48.885924 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28316f48-12e7-4420-a787-a885f55fe844 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:48.888138 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28316f48-12e7-4420-a787-a885f55fe844 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:48.901781 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1092/2203] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:43:48.930729 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1112/2204] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:43:48 2022] POST /identity/v3/users => generated 288 bytes in 523 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:43:48.934622 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28316f48-12e7-4420-a787-a885f55fe844 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:48.950141 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28316f48-12e7-4420-a787-a885f55fe844 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:48.957414 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6ad14-cb3d-46d4-a201-d7c82404c958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:48.958306 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6ad14-cb3d-46d4-a201-d7c82404c958 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:48.958807 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6ad14-cb3d-46d4-a201-d7c82404c958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:48.959419 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6ad14-cb3d-46d4-a201-d7c82404c958 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:48.960442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6ad14-cb3d-46d4-a201-d7c82404c958 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:48.962353 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6ad14-cb3d-46d4-a201-d7c82404c958 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:48.977547 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1093/2205] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:43:48.982891 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5150d39-9db3-486a-9c96-07d9ee2df160 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:48.998588 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5150d39-9db3-486a-9c96-07d9ee2df160 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:49.004281 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9a88c-b460-496f-99cc-45f2eea349ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:49.005130 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9a88c-b460-496f-99cc-45f2eea349ab admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:49.006339 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9a88c-b460-496f-99cc-45f2eea349ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:49.006339 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9a88c-b460-496f-99cc-45f2eea349ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:49.006656 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc9a88c-b460-496f-99cc-45f2eea349ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:49.008594 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc9a88c-b460-496f-99cc-45f2eea349ab admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:49.025880 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1113/2206] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:43:49.030883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6ad14-cb3d-46d4-a201-d7c82404c958 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:49.044480 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6ad14-cb3d-46d4-a201-d7c82404c958 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:49.050484 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2464f-7683-468f-b345-4cfba4f950f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:49.051284 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2464f-7683-468f-b345-4cfba4f950f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:49.051821 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2464f-7683-468f-b345-4cfba4f950f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:49.052173 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2464f-7683-468f-b345-4cfba4f950f6 admin admin] PATH_INFO: `/v3/projects/385f46cc26a74b9d93eecb72357b4c98/users/b1950f82068c4fbbab0c65dd968581e1/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:49.068926 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd2464f-7683-468f-b345-4cfba4f950f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=385f46cc26a74b9d93eecb72357b4c98, user_id=b1950f82068c4fbbab0c65dd968581e1, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:49.071767 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd2464f-7683-468f-b345-4cfba4f950f6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:49.152476 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1094/2207] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:43:49 2022] PUT /identity/v3/projects/385f46cc26a74b9d93eecb72357b4c98/users/b1950f82068c4fbbab0c65dd968581e1/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:49.163670 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20e9b7-f85c-45b8-8d95-b2902e82301f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:49.163968 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20e9b7-f85c-45b8-8d95-b2902e82301f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:49.164776 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20e9b7-f85c-45b8-8d95-b2902e82301f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:49.173858 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8621222-c95a-4577-9e1d-97a56170aac3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:49.174199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8621222-c95a-4577-9e1d-97a56170aac3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:49.174545 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8621222-c95a-4577-9e1d-97a56170aac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:49.195398 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ce20e9b7-f85c-45b8-8d95-b2902e82301f None None] MFA Rules not processed for user `b1950f82068c4fbbab0c65dd968581e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:49.196711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ce20e9b7-f85c-45b8-8d95-b2902e82301f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:49.211300 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a8621222-c95a-4577-9e1d-97a56170aac3 None None] MFA Rules not processed for user `e42059e9af9842548ea08dcd39ff0b6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:49.212630 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a8621222-c95a-4577-9e1d-97a56170aac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:49.289397 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1114/2208] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:49.294374 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1095/2209] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 10 12:43:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:49.312939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20e9b7-f85c-45b8-8d95-b2902e82301f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:49.315726 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ce20e9b7-f85c-45b8-8d95-b2902e82301f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:49.360244 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8621222-c95a-4577-9e1d-97a56170aac3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:49.362976 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a8621222-c95a-4577-9e1d-97a56170aac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:49.397211 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20e9b7-f85c-45b8-8d95-b2902e82301f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:49.401537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdcf635-e64f-4352-9d2f-488f79e8da5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:49.402973 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcf635-e64f-4352-9d2f-488f79e8da5c service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:49.403543 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcf635-e64f-4352-9d2f-488f79e8da5c service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:49.404624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcf635-e64f-4352-9d2f-488f79e8da5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:49.434296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdcf635-e64f-4352-9d2f-488f79e8da5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:49.434296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdcf635-e64f-4352-9d2f-488f79e8da5c service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:49.434296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:49.439370 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1115/2210] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:43:49.471723 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52fdd0-010e-4115-bd13-3157e5ee0fb5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:49.472018 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52fdd0-010e-4115-bd13-3157e5ee0fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:49.472880 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52fdd0-010e-4115-bd13-3157e5ee0fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:49.492703 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:49.513611 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7d52fdd0-010e-4115-bd13-3157e5ee0fb5 None None] MFA Rules not processed for user `00b19e56e11047e3b0f102efea439b7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:49.514987 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7d52fdd0-010e-4115-bd13-3157e5ee0fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:49.515392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8621222-c95a-4577-9e1d-97a56170aac3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:49.522877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d5b33-a33f-4a1a-b482-20555aaaa8d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59395bfa505c4540b3957deee78eb698', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:49.523598 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d5b33-a33f-4a1a-b482-20555aaaa8d1 service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:49.523884 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d5b33-a33f-4a1a-b482-20555aaaa8d1 service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:49.524161 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d5b33-a33f-4a1a-b482-20555aaaa8d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:49.543746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d5b33-a33f-4a1a-b482-20555aaaa8d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:49.553076 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d5b33-a33f-4a1a-b482-20555aaaa8d1 service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:49.589296 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1096/2211] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 12:43:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 12:43:49.645003 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1116/2212] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:43:49.801235 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d5b33-a33f-4a1a-b482-20555aaaa8d1 service swift] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:49.807793 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-923d5b33-a33f-4a1a-b482-20555aaaa8d1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:49.924185 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d5b33-a33f-4a1a-b482-20555aaaa8d1 service swift] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:49.928629 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70776546-13e6-47a9-b796-5698b2fdcb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:49.929472 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70776546-13e6-47a9-b796-5698b2fdcb1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:49.929880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70776546-13e6-47a9-b796-5698b2fdcb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:49.930386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70776546-13e6-47a9-b796-5698b2fdcb1d admin admin] PATH_INFO: `/v3/projects/98abede88c5c4949ab7a373def1d2f7a` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:49.931895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70776546-13e6-47a9-b796-5698b2fdcb1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=98abede88c5c4949ab7a373def1d2f7a)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:49.933976 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70776546-13e6-47a9-b796-5698b2fdcb1d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:50.107431 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-70776546-13e6-47a9-b796-5698b2fdcb1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98abede88c5c4949ab7a373def1d2f7a', 'request_id': 'req-70776546-13e6-47a9-b796-5698b2fdcb1d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:50.123261 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-70776546-13e6-47a9-b796-5698b2fdcb1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98abede88c5c4949ab7a373def1d2f7a', 'request_id': 'req-70776546-13e6-47a9-b796-5698b2fdcb1d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:50.237611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-70776546-13e6-47a9-b796-5698b2fdcb1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98abede88c5c4949ab7a373def1d2f7a', 'request_id': 'req-70776546-13e6-47a9-b796-5698b2fdcb1d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:50.252014 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-70776546-13e6-47a9-b796-5698b2fdcb1d admin admin] The token cache is being invalidate because project 98abede88c5c4949ab7a373def1d2f7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:43:50.252313 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-70776546-13e6-47a9-b796-5698b2fdcb1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:50.253456 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-70776546-13e6-47a9-b796-5698b2fdcb1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:50.267592 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1097/2213] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:43:49 2022] DELETE /identity/v3/projects/98abede88c5c4949ab7a373def1d2f7a => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:50.274480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9486c8c-4291-4085-a92a-1b599c168b94 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:50.274988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9486c8c-4291-4085-a92a-1b599c168b94 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:50.274988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9486c8c-4291-4085-a92a-1b599c168b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:50.326889 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b9486c8c-4291-4085-a92a-1b599c168b94 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:50.331729 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b9486c8c-4291-4085-a92a-1b599c168b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:50.501276 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19025f6-f2cd-442a-ac0c-919b96e4ab37 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:50.501621 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19025f6-f2cd-442a-ac0c-919b96e4ab37 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:50.501839 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19025f6-f2cd-442a-ac0c-919b96e4ab37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:50.507349 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1117/2214] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:50.575737 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a19025f6-f2cd-442a-ac0c-919b96e4ab37 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:50.576994 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a19025f6-f2cd-442a-ac0c-919b96e4ab37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:50.617382 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1098/2215] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:50.621970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9486c8c-4291-4085-a92a-1b599c168b94 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:50.642984 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9486c8c-4291-4085-a92a-1b599c168b94 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:50.668558 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17932920-470f-4ec4-8239-ddc3b82e1722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:50.669226 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17932920-470f-4ec4-8239-ddc3b82e1722 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:50.669581 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17932920-470f-4ec4-8239-ddc3b82e1722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:50.669771 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17932920-470f-4ec4-8239-ddc3b82e1722 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:50.670443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17932920-470f-4ec4-8239-ddc3b82e1722 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:43:50.670754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17932920-470f-4ec4-8239-ddc3b82e1722 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:50.672478 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17932920-470f-4ec4-8239-ddc3b82e1722 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:50.713905 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1118/2216] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:43:50.715819 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19025f6-f2cd-442a-ac0c-919b96e4ab37 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:50.738633 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19025f6-f2cd-442a-ac0c-919b96e4ab37 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:50.752061 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11816c58-b42b-4cea-bc3b-731f54bd84c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:50.752705 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11816c58-b42b-4cea-bc3b-731f54bd84c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:50.752992 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11816c58-b42b-4cea-bc3b-731f54bd84c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:50.753267 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11816c58-b42b-4cea-bc3b-731f54bd84c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:50.753908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11816c58-b42b-4cea-bc3b-731f54bd84c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:50.755884 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11816c58-b42b-4cea-bc3b-731f54bd84c2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:50.911248 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1099/2217] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:43:50 2022] POST /identity/v3/projects => generated 381 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:43:50.915427 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17932920-470f-4ec4-8239-ddc3b82e1722 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:50.928582 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17932920-470f-4ec4-8239-ddc3b82e1722 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:51.069663 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af395ba8-a26e-4cae-b4a8-7de194a2fe59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:51.070388 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af395ba8-a26e-4cae-b4a8-7de194a2fe59 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:51.070737 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af395ba8-a26e-4cae-b4a8-7de194a2fe59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:51.074119 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af395ba8-a26e-4cae-b4a8-7de194a2fe59 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:51.074634 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af395ba8-a26e-4cae-b4a8-7de194a2fe59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:51.080137 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af395ba8-a26e-4cae-b4a8-7de194a2fe59 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:51.206626 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1119/2218] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:43:50 2022] POST /identity/v3/users => generated 291 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:43:51.210510 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11816c58-b42b-4cea-bc3b-731f54bd84c2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:51.225721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11816c58-b42b-4cea-bc3b-731f54bd84c2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:51.271910 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e9e12-b701-433b-8b9b-9c217e03fbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:51.272544 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e9e12-b701-433b-8b9b-9c217e03fbdf admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:51.272828 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e9e12-b701-433b-8b9b-9c217e03fbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:51.273105 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e9e12-b701-433b-8b9b-9c217e03fbdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:51.273762 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e9e12-b701-433b-8b9b-9c217e03fbdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:51.292864 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e9e12-b701-433b-8b9b-9c217e03fbdf admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:51.345556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1100/2219] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:43:51.347082 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af395ba8-a26e-4cae-b4a8-7de194a2fe59 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:51.366776 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af395ba8-a26e-4cae-b4a8-7de194a2fe59 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:51.392601 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f56ade-8287-4c47-8290-ec0719488a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:51.394263 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f56ade-8287-4c47-8290-ec0719488a1f admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:51.394364 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f56ade-8287-4c47-8290-ec0719488a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:51.394808 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f56ade-8287-4c47-8290-ec0719488a1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:51.395577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f56ade-8287-4c47-8290-ec0719488a1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:51.397288 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f56ade-8287-4c47-8290-ec0719488a1f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:51.432123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1120/2220] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:43:51.436368 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e9e12-b701-433b-8b9b-9c217e03fbdf admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:51.464327 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e9e12-b701-433b-8b9b-9c217e03fbdf admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:51.468987 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587b644-8bc8-49b9-8bae-0321abbdbce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:51.469623 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587b644-8bc8-49b9-8bae-0321abbdbce3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:51.470002 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587b644-8bc8-49b9-8bae-0321abbdbce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:51.470363 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587b644-8bc8-49b9-8bae-0321abbdbce3 admin admin] PATH_INFO: `/v3/projects/4f4fa99fc6514cceb31af25c94623178/users/d8e3d29dcfbc43a1aa2c26b676f5894c/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:51.509366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4587b644-8bc8-49b9-8bae-0321abbdbce3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f4fa99fc6514cceb31af25c94623178, user_id=d8e3d29dcfbc43a1aa2c26b676f5894c, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:51.513641 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4587b644-8bc8-49b9-8bae-0321abbdbce3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:51.877412 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1101/2221] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:43:51 2022] PUT /identity/v3/projects/4f4fa99fc6514cceb31af25c94623178/users/d8e3d29dcfbc43a1aa2c26b676f5894c/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:51.878938 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d0c12-a06a-40d2-92f8-52780faf96d5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:51.879255 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d0c12-a06a-40d2-92f8-52780faf96d5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:51.879539 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d0c12-a06a-40d2-92f8-52780faf96d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:51.926242 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-df7d0c12-a06a-40d2-92f8-52780faf96d5 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:51.932601 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-df7d0c12-a06a-40d2-92f8-52780faf96d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:51.952417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:52.073365 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1121/2222] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:52.110929 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587b644-8bc8-49b9-8bae-0321abbdbce3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:52.113310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4587b644-8bc8-49b9-8bae-0321abbdbce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:52.141875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:52.254762 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587b644-8bc8-49b9-8bae-0321abbdbce3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:52.268273 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e34c15-a84c-4a3d-aa97-cc84c0a1d54d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:52.269737 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e34c15-a84c-4a3d-aa97-cc84c0a1d54d service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:52.271343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e34c15-a84c-4a3d-aa97-cc84c0a1d54d service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:52.271711 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e34c15-a84c-4a3d-aa97-cc84c0a1d54d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:52.298978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e34c15-a84c-4a3d-aa97-cc84c0a1d54d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:52.300722 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e34c15-a84c-4a3d-aa97-cc84c0a1d54d service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:52.322637 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1102/2223] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:43:54.294383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9a9ea-b651-4c2b-a3bb-e3bf8ccc278a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:54.294989 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9a9ea-b651-4c2b-a3bb-e3bf8ccc278a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:54.294989 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9a9ea-b651-4c2b-a3bb-e3bf8ccc278a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:54.332735 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-bdf9a9ea-b651-4c2b-a3bb-e3bf8ccc278a None None] MFA Rules not processed for user `d8e3d29dcfbc43a1aa2c26b676f5894c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:54.334925 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bdf9a9ea-b651-4c2b-a3bb-e3bf8ccc278a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:54.433851 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1122/2224] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:43:54.465832 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e34c15-a84c-4a3d-aa97-cc84c0a1d54d service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:54.467290 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b1e34c15-a84c-4a3d-aa97-cc84c0a1d54d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:54.528609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e34c15-a84c-4a3d-aa97-cc84c0a1d54d service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:54.533511 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4576b369-a6bd-4ba0-a5cb-67d748bb013c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:54.534307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576b369-a6bd-4ba0-a5cb-67d748bb013c service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:54.534583 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576b369-a6bd-4ba0-a5cb-67d748bb013c service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:54.534904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576b369-a6bd-4ba0-a5cb-67d748bb013c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:54.548754 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4576b369-a6bd-4ba0-a5cb-67d748bb013c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:54.550242 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4576b369-a6bd-4ba0-a5cb-67d748bb013c service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:54.559651 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1103/2225] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:43:56.511913 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60a7a0-734d-4ef3-86ec-071ec6f12efe None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:56.512918 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60a7a0-734d-4ef3-86ec-071ec6f12efe None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:56.513381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60a7a0-734d-4ef3-86ec-071ec6f12efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:56.534083 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-cb60a7a0-734d-4ef3-86ec-071ec6f12efe None None] MFA Rules not processed for user `d8e3d29dcfbc43a1aa2c26b676f5894c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:43:56.535638 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cb60a7a0-734d-4ef3-86ec-071ec6f12efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:56.550039 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1123/2226] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:43:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:44:04.558617 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=46 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:44:06.528098 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=58 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:44:15.515680 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4576b369-a6bd-4ba0-a5cb-67d748bb013c service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:15.517166 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4576b369-a6bd-4ba0-a5cb-67d748bb013c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:15.533961 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4576b369-a6bd-4ba0-a5cb-67d748bb013c service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:15.538209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6fc43-2c41-453c-844f-5265a6b6b794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:15.538830 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6fc43-2c41-453c-844f-5265a6b6b794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:15.539109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6fc43-2c41-453c-844f-5265a6b6b794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:15.539427 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6fc43-2c41-453c-844f-5265a6b6b794 admin admin] PATH_INFO: `/v3/users/b1950f82068c4fbbab0c65dd968581e1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:15.540574 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb6fc43-2c41-453c-844f-5265a6b6b794 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1950f82068c4fbbab0c65dd968581e1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:15.542648 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb6fc43-2c41-453c-844f-5265a6b6b794 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:15.710727 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1cb6fc43-2c41-453c-844f-5265a6b6b794 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1950f82068c4fbbab0c65dd968581e1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:15.824885 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1cb6fc43-2c41-453c-844f-5265a6b6b794 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1950f82068c4fbbab0c65dd968581e1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:15.828436 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1cb6fc43-2c41-453c-844f-5265a6b6b794 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1950f82068c4fbbab0c65dd968581e1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:15.828436 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1cb6fc43-2c41-453c-844f-5265a6b6b794 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1950f82068c4fbbab0c65dd968581e1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:15.837639 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1cb6fc43-2c41-453c-844f-5265a6b6b794 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1950f82068c4fbbab0c65dd968581e1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:15.855759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1104/2227] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:44:15 2022] DELETE /identity/v3/users/b1950f82068c4fbbab0c65dd968581e1 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:44:15.860896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30e313-4d32-41f9-85ce-9aac514bd5c0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:15.861375 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30e313-4d32-41f9-85ce-9aac514bd5c0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:15.861470 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30e313-4d32-41f9-85ce-9aac514bd5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:15.882519 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7b30e313-4d32-41f9-85ce-9aac514bd5c0 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:44:15.883755 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7b30e313-4d32-41f9-85ce-9aac514bd5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:15.947569 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1124/2228] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:44:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:44:15.960079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6fc43-2c41-453c-844f-5265a6b6b794 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:15.961608 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1cb6fc43-2c41-453c-844f-5265a6b6b794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:16.019596 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6fc43-2c41-453c-844f-5265a6b6b794 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:16.023495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f1ab2-7798-48ae-b6e7-1033c6901801 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:16.024106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f1ab2-7798-48ae-b6e7-1033c6901801 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:16.024387 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f1ab2-7798-48ae-b6e7-1033c6901801 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:16.024657 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f1ab2-7798-48ae-b6e7-1033c6901801 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:16.033664 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f1ab2-7798-48ae-b6e7-1033c6901801 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:16.035113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f1ab2-7798-48ae-b6e7-1033c6901801 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:16.044353 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1105/2229] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:44:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:44:16.438203 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30e313-4d32-41f9-85ce-9aac514bd5c0 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:16.439747 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7b30e313-4d32-41f9-85ce-9aac514bd5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:16.452179 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30e313-4d32-41f9-85ce-9aac514bd5c0 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:16.458238 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0488a303-c82d-4faf-b7a4-30da34f1de16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:16.458945 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488a303-c82d-4faf-b7a4-30da34f1de16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:16.459451 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488a303-c82d-4faf-b7a4-30da34f1de16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:16.459777 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488a303-c82d-4faf-b7a4-30da34f1de16 admin admin] PATH_INFO: `/v3/projects/385f46cc26a74b9d93eecb72357b4c98` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:16.461131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0488a303-c82d-4faf-b7a4-30da34f1de16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=385f46cc26a74b9d93eecb72357b4c98)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:16.463130 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0488a303-c82d-4faf-b7a4-30da34f1de16 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:16.586965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0488a303-c82d-4faf-b7a4-30da34f1de16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '385f46cc26a74b9d93eecb72357b4c98', 'request_id': 'req-0488a303-c82d-4faf-b7a4-30da34f1de16'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:16.595235 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0488a303-c82d-4faf-b7a4-30da34f1de16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '385f46cc26a74b9d93eecb72357b4c98', 'request_id': 'req-0488a303-c82d-4faf-b7a4-30da34f1de16'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:16.740114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0488a303-c82d-4faf-b7a4-30da34f1de16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '385f46cc26a74b9d93eecb72357b4c98', 'request_id': 'req-0488a303-c82d-4faf-b7a4-30da34f1de16'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:16.752775 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0488a303-c82d-4faf-b7a4-30da34f1de16 admin admin] The token cache is being invalidate because project 385f46cc26a74b9d93eecb72357b4c98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:44:16.753068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0488a303-c82d-4faf-b7a4-30da34f1de16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:16.753704 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0488a303-c82d-4faf-b7a4-30da34f1de16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:16.755269 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1125/2230] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:44:16 2022] DELETE /identity/v3/projects/385f46cc26a74b9d93eecb72357b4c98 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:44:16.760384 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87146b-f026-46df-a389-3ab83c67fa94 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:16.760839 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87146b-f026-46df-a389-3ab83c67fa94 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:16.760930 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87146b-f026-46df-a389-3ab83c67fa94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:16.781589 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-bc87146b-f026-46df-a389-3ab83c67fa94 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:44:16.782846 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bc87146b-f026-46df-a389-3ab83c67fa94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:16.841835 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1106/2231] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:44:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:44:16.849928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f7569-6027-43cf-ad27-b59308c6a342 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:16.850216 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f7569-6027-43cf-ad27-b59308c6a342 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:16.850517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f7569-6027-43cf-ad27-b59308c6a342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:16.871112 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b93f7569-6027-43cf-ad27-b59308c6a342 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:44:16.872351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b93f7569-6027-43cf-ad27-b59308c6a342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:16.881246 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1126/2232] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:44:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:44:16.885763 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87146b-f026-46df-a389-3ab83c67fa94 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:16.896403 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87146b-f026-46df-a389-3ab83c67fa94 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:16.901456 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e11d3-d6ce-4738-82a9-3e04a27047f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:16.905397 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e11d3-d6ce-4738-82a9-3e04a27047f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:16.905397 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e11d3-d6ce-4738-82a9-3e04a27047f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:16.905397 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e11d3-d6ce-4738-82a9-3e04a27047f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:16.905397 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e11d3-d6ce-4738-82a9-3e04a27047f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:44:16.905397 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e11d3-d6ce-4738-82a9-3e04a27047f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:16.905397 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e11d3-d6ce-4738-82a9-3e04a27047f4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:16.928069 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1107/2233] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:44:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:44:16.932593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f7569-6027-43cf-ad27-b59308c6a342 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:16.943225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f7569-6027-43cf-ad27-b59308c6a342 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:16.948039 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0959e-afe5-471c-9a20-887ba48f765b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:16.951461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0959e-afe5-471c-9a20-887ba48f765b admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:16.951461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0959e-afe5-471c-9a20-887ba48f765b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:16.951461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0959e-afe5-471c-9a20-887ba48f765b admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:16.951461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e0959e-afe5-471c-9a20-887ba48f765b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:16.951799 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e0959e-afe5-471c-9a20-887ba48f765b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:17.062636 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a4d5b-2dc7-4077-a3e4-31cc63784e5f None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:17.062846 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a4d5b-2dc7-4077-a3e4-31cc63784e5f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:17.063155 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a4d5b-2dc7-4077-a3e4-31cc63784e5f None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:17.064582 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1108/2234] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 10 12:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:44:17.071744 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f389b6-357c-48dc-817c-d5e52974d4fd None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:17.072468 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f389b6-357c-48dc-817c-d5e52974d4fd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:17.073247 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f389b6-357c-48dc-817c-d5e52974d4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:17.087614 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1127/2235] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:44:16 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:44:17.091690 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0959e-afe5-471c-9a20-887ba48f765b admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:17.096503 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:44:17.115545 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b1f389b6-357c-48dc-817c-d5e52974d4fd None None] MFA Rules not processed for user `eb3f6deffb54467f8b923956ed2a6acf`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:44:17.117259 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b1f389b6-357c-48dc-817c-d5e52974d4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:17.232386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1109/2236] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 10 12:44:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:44:17.246306 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f389b6-357c-48dc-817c-d5e52974d4fd None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:17.249704 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b1f389b6-357c-48dc-817c-d5e52974d4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:17.249834 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:44:17.265896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0959e-afe5-471c-9a20-887ba48f765b admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:17.271946 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff725f57-d64a-42aa-9c71-585a2edba5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:17.273317 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff725f57-d64a-42aa-9c71-585a2edba5ff admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:17.273317 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff725f57-d64a-42aa-9c71-585a2edba5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:17.273317 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff725f57-d64a-42aa-9c71-585a2edba5ff admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:17.273944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff725f57-d64a-42aa-9c71-585a2edba5ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:17.280681 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff725f57-d64a-42aa-9c71-585a2edba5ff admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:17.317765 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f389b6-357c-48dc-817c-d5e52974d4fd None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:17.322175 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f055e0-b11a-46e8-86ae-8fcc5a306344 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35dd27fd356241b885706375d880222f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:17.322789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f055e0-b11a-46e8-86ae-8fcc5a306344 service glance] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:17.323220 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f055e0-b11a-46e8-86ae-8fcc5a306344 service glance] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:17.323664 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f055e0-b11a-46e8-86ae-8fcc5a306344 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:17.335653 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f055e0-b11a-46e8-86ae-8fcc5a306344 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:17.340642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f055e0-b11a-46e8-86ae-8fcc5a306344 service glance] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:17.349363 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1110/2237] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:44:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:44:17.363759 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1128/2238] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:44:17 2022] POST /identity/v3/users => generated 287 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:44:17.368514 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f055e0-b11a-46e8-86ae-8fcc5a306344 service glance] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:17.380601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f055e0-b11a-46e8-86ae-8fcc5a306344 service glance] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:17.385589 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf15fb-5036-4947-83b7-9e4b3482edb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:17.387885 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf15fb-5036-4947-83b7-9e4b3482edb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:17.387885 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf15fb-5036-4947-83b7-9e4b3482edb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:17.387885 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf15fb-5036-4947-83b7-9e4b3482edb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:17.387885 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf15fb-5036-4947-83b7-9e4b3482edb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:17.389293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf15fb-5036-4947-83b7-9e4b3482edb2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:17.403690 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1111/2239] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:44:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:44:17.404939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff725f57-d64a-42aa-9c71-585a2edba5ff admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:17.415461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff725f57-d64a-42aa-9c71-585a2edba5ff admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:17.420263 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e30aa-fde5-4e8c-a19b-80707f443812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:17.420862 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e30aa-fde5-4e8c-a19b-80707f443812 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:17.421145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e30aa-fde5-4e8c-a19b-80707f443812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:17.421955 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e30aa-fde5-4e8c-a19b-80707f443812 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:17.422644 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e30aa-fde5-4e8c-a19b-80707f443812 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:17.424421 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e30aa-fde5-4e8c-a19b-80707f443812 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:17.435147 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1129/2240] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:44:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:44:17.440100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf15fb-5036-4947-83b7-9e4b3482edb2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:17.451458 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf15fb-5036-4947-83b7-9e4b3482edb2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:17.456192 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ce763-eca7-4a74-af35-833a3360ec02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:17.457138 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ce763-eca7-4a74-af35-833a3360ec02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:17.457138 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ce763-eca7-4a74-af35-833a3360ec02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:17.457381 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ce763-eca7-4a74-af35-833a3360ec02 admin admin] PATH_INFO: `/v3/projects/68f7eac0130f408eb421a83192aae2dd/users/a3b27ba192594da1b85501d9108d0698/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:17.472945 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ce763-eca7-4a74-af35-833a3360ec02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68f7eac0130f408eb421a83192aae2dd, user_id=a3b27ba192594da1b85501d9108d0698, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:17.475712 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ce763-eca7-4a74-af35-833a3360ec02 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:17.570932 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ecc056-4810-4bb3-b795-8e47bf328070 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:17.571858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ecc056-4810-4bb3-b795-8e47bf328070 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:17.571858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ecc056-4810-4bb3-b795-8e47bf328070 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:17.573074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1130/2241] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 10 12:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:44:17.577988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99dbeb-d347-4020-b5b6-9e402cf31308 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:17.578401 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99dbeb-d347-4020-b5b6-9e402cf31308 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:17.578753 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99dbeb-d347-4020-b5b6-9e402cf31308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:17.601080 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1112/2242] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:44:17 2022] PUT /identity/v3/projects/68f7eac0130f408eb421a83192aae2dd/users/a3b27ba192594da1b85501d9108d0698/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:44:17.601270 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-1d99dbeb-d347-4020-b5b6-9e402cf31308 None None] MFA Rules not processed for user `5db814952f9f466abf6148b7a0db2173`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:44:17.602144 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1d99dbeb-d347-4020-b5b6-9e402cf31308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:17.675483 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b6868-00b4-44d1-bf46-04949cc935ce None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:17.675777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b6868-00b4-44d1-bf46-04949cc935ce None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:17.676542 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b6868-00b4-44d1-bf46-04949cc935ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:17.693451 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1131/2243] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 10 12:44:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:44:17.707433 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99dbeb-d347-4020-b5b6-9e402cf31308 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:17.709183 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1d99dbeb-d347-4020-b5b6-9e402cf31308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:17.720434 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3c9b6868-00b4-44d1-bf46-04949cc935ce None None] MFA Rules not processed for user `a3b27ba192594da1b85501d9108d0698`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:44:17.725591 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3c9b6868-00b4-44d1-bf46-04949cc935ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:17.725735 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99dbeb-d347-4020-b5b6-9e402cf31308 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:17.728727 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154563e-0e6b-4aff-ad3a-358329028c0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:17.729543 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154563e-0e6b-4aff-ad3a-358329028c0a service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:17.729971 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154563e-0e6b-4aff-ad3a-358329028c0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:17.730671 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154563e-0e6b-4aff-ad3a-358329028c0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:17.741003 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154563e-0e6b-4aff-ad3a-358329028c0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:17.743654 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154563e-0e6b-4aff-ad3a-358329028c0a service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:17.757063 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1132/2244] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:44:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:44:17.796167 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:44:17.827314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1113/2245] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:44:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:44:17.838548 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154563e-0e6b-4aff-ad3a-358329028c0a service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:17.845493 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6154563e-0e6b-4aff-ad3a-358329028c0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:17.865692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2af749-7157-4f31-8214-2c734b350aab None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:17.865978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2af749-7157-4f31-8214-2c734b350aab None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:17.866257 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2af749-7157-4f31-8214-2c734b350aab None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:17.867955 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1114/2246] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 10 12:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:44:17.873821 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4a821-0b70-4780-a02d-643c4e939df9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:17.874184 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4a821-0b70-4780-a02d-643c4e939df9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:17.874473 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4a821-0b70-4780-a02d-643c4e939df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:17.897831 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c9f4a821-0b70-4780-a02d-643c4e939df9 None None] MFA Rules not processed for user `eb3f6deffb54467f8b923956ed2a6acf`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:44:17.899460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c9f4a821-0b70-4780-a02d-643c4e939df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:17.910995 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154563e-0e6b-4aff-ad3a-358329028c0a service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:17.914752 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb19e5-0824-4077-9a4e-fbcb589ac1c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:17.915930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb19e5-0824-4077-9a4e-fbcb589ac1c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:17.915930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb19e5-0824-4077-9a4e-fbcb589ac1c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:17.916132 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb19e5-0824-4077-9a4e-fbcb589ac1c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:17.930314 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb19e5-0824-4077-9a4e-fbcb589ac1c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:17.932378 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb19e5-0824-4077-9a4e-fbcb589ac1c0 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:17.942900 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1133/2247] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:44:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:44:17.980357 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb19e5-0824-4077-9a4e-fbcb589ac1c0 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:17.996071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1115/2248] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 10 12:44:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:44:18.008769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4a821-0b70-4780-a02d-643c4e939df9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:18.021651 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4a821-0b70-4780-a02d-643c4e939df9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:18.026938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f5b3d-54e5-4d1b-853d-eb5116e1fa3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:18.027728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f5b3d-54e5-4d1b-853d-eb5116e1fa3a service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:18.028081 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f5b3d-54e5-4d1b-853d-eb5116e1fa3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:18.028423 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f5b3d-54e5-4d1b-853d-eb5116e1fa3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:18.042607 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f5b3d-54e5-4d1b-853d-eb5116e1fa3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:18.044174 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f5b3d-54e5-4d1b-853d-eb5116e1fa3a service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:18.056933 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1116/2249] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:44:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:44:18.081539 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb19e5-0824-4077-9a4e-fbcb589ac1c0 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:18.086642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8ad22-4d3c-4bf7-8ea9-2f6009c5d3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:18.087398 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8ad22-4d3c-4bf7-8ea9-2f6009c5d3ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:18.087764 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8ad22-4d3c-4bf7-8ea9-2f6009c5d3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:18.088170 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8ad22-4d3c-4bf7-8ea9-2f6009c5d3ed admin admin] PATH_INFO: `/v3/users/a3b27ba192594da1b85501d9108d0698` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:18.089398 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e8ad22-4d3c-4bf7-8ea9-2f6009c5d3ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3b27ba192594da1b85501d9108d0698)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:18.091548 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e8ad22-4d3c-4bf7-8ea9-2f6009c5d3ed admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:18.113394 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:44:18.311254 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-07e8ad22-4d3c-4bf7-8ea9-2f6009c5d3ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3b27ba192594da1b85501d9108d0698'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:18.319756 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-07e8ad22-4d3c-4bf7-8ea9-2f6009c5d3ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3b27ba192594da1b85501d9108d0698'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:18.413081 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-07e8ad22-4d3c-4bf7-8ea9-2f6009c5d3ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3b27ba192594da1b85501d9108d0698'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:18.413907 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-07e8ad22-4d3c-4bf7-8ea9-2f6009c5d3ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3b27ba192594da1b85501d9108d0698'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:18.414754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-07e8ad22-4d3c-4bf7-8ea9-2f6009c5d3ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3b27ba192594da1b85501d9108d0698'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:18.440070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1134/2250] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:44:17 2022] DELETE /identity/v3/users/a3b27ba192594da1b85501d9108d0698 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:44:18.444880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ad6b0-1314-4aec-8786-b3d8784d980b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:18.445167 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ad6b0-1314-4aec-8786-b3d8784d980b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:18.445444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ad6b0-1314-4aec-8786-b3d8784d980b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:18.470250 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f82ad6b0-1314-4aec-8786-b3d8784d980b None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:44:18.471495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f82ad6b0-1314-4aec-8786-b3d8784d980b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:18.537496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1117/2251] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:44:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:44:18.544100 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8ad22-4d3c-4bf7-8ea9-2f6009c5d3ed admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:18.545587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-07e8ad22-4d3c-4bf7-8ea9-2f6009c5d3ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:18.611743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8ad22-4d3c-4bf7-8ea9-2f6009c5d3ed admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:18.615653 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829a6ca2-948b-4fc9-a682-181f2ecca698 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:18.616266 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a6ca2-948b-4fc9-a682-181f2ecca698 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:18.616562 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a6ca2-948b-4fc9-a682-181f2ecca698 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:18.616837 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a6ca2-948b-4fc9-a682-181f2ecca698 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:18.626363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829a6ca2-948b-4fc9-a682-181f2ecca698 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:18.627834 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829a6ca2-948b-4fc9-a682-181f2ecca698 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:18.637493 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1135/2252] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:44:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:44:19.020797 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ad6b0-1314-4aec-8786-b3d8784d980b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:19.022266 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f82ad6b0-1314-4aec-8786-b3d8784d980b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:19.036029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ad6b0-1314-4aec-8786-b3d8784d980b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:19.041655 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2b76d-c631-4ee2-b74b-f4022b28e37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:19.042303 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2b76d-c631-4ee2-b74b-f4022b28e37d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:19.042584 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2b76d-c631-4ee2-b74b-f4022b28e37d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:19.042857 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2b76d-c631-4ee2-b74b-f4022b28e37d admin admin] PATH_INFO: `/v3/projects/68f7eac0130f408eb421a83192aae2dd` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:19.044031 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f2b76d-c631-4ee2-b74b-f4022b28e37d admin admin] RBAC: Authorizing `identity:delete_project(project_id=68f7eac0130f408eb421a83192aae2dd)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:19.045932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f2b76d-c631-4ee2-b74b-f4022b28e37d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:19.153186 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-08f2b76d-c631-4ee2-b74b-f4022b28e37d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68f7eac0130f408eb421a83192aae2dd', 'request_id': 'req-08f2b76d-c631-4ee2-b74b-f4022b28e37d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:19.159529 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-08f2b76d-c631-4ee2-b74b-f4022b28e37d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68f7eac0130f408eb421a83192aae2dd', 'request_id': 'req-08f2b76d-c631-4ee2-b74b-f4022b28e37d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:19.237393 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-08f2b76d-c631-4ee2-b74b-f4022b28e37d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68f7eac0130f408eb421a83192aae2dd', 'request_id': 'req-08f2b76d-c631-4ee2-b74b-f4022b28e37d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:19.238796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:44:19.251104 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-08f2b76d-c631-4ee2-b74b-f4022b28e37d admin admin] The token cache is being invalidate because project 68f7eac0130f408eb421a83192aae2dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:44:19.251433 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-08f2b76d-c631-4ee2-b74b-f4022b28e37d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:19.252158 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-08f2b76d-c631-4ee2-b74b-f4022b28e37d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:19.253663 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1118/2253] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:44:19 2022] DELETE /identity/v3/projects/68f7eac0130f408eb421a83192aae2dd => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:44:24.169291 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938b04c-9f90-4ee9-ae61-91f646ca9989 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:24.169734 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938b04c-9f90-4ee9-ae61-91f646ca9989 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:24.169821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938b04c-9f90-4ee9-ae61-91f646ca9989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:24.188707 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-4938b04c-9f90-4ee9-ae61-91f646ca9989 None None] MFA Rules not processed for user `e42059e9af9842548ea08dcd39ff0b6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:44:24.190008 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4938b04c-9f90-4ee9-ae61-91f646ca9989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:24.246302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1136/2254] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 10 12:44:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:44:24.260492 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2b76d-c631-4ee2-b74b-f4022b28e37d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:24.260492 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-08f2b76d-c631-4ee2-b74b-f4022b28e37d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:24.330901 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2b76d-c631-4ee2-b74b-f4022b28e37d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:24.336270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c21eae-8b35-4f1a-a098-615d3f87d8e3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59395bfa505c4540b3957deee78eb698', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:24.336885 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c21eae-8b35-4f1a-a098-615d3f87d8e3 service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:24.337186 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c21eae-8b35-4f1a-a098-615d3f87d8e3 service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:24.337455 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c21eae-8b35-4f1a-a098-615d3f87d8e3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:24.346610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c21eae-8b35-4f1a-a098-615d3f87d8e3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:24.348215 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c21eae-8b35-4f1a-a098-615d3f87d8e3 service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:24.356098 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1119/2255] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 12:44:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 12:44:34.239404 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=72,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:44:34.352911 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=35 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:44:35.090345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d581fd0-de92-4c23-b622-bba7ca658faa None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:35.090635 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d581fd0-de92-4c23-b622-bba7ca658faa None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:35.090937 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d581fd0-de92-4c23-b622-bba7ca658faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:35.114023 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6d581fd0-de92-4c23-b622-bba7ca658faa None None] MFA Rules not processed for user `e42059e9af9842548ea08dcd39ff0b6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:44:35.115312 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6d581fd0-de92-4c23-b622-bba7ca658faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:35.123071 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1137/2256] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 10 12:44:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:44:35.145533 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c21eae-8b35-4f1a-a098-615d3f87d8e3 service swift] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:35.165156 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c21eae-8b35-4f1a-a098-615d3f87d8e3 service swift] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:35.170862 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725ddd3-7371-49ff-82c3-ab10e1fc283e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59395bfa505c4540b3957deee78eb698', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:35.171829 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725ddd3-7371-49ff-82c3-ab10e1fc283e service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:35.171938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725ddd3-7371-49ff-82c3-ab10e1fc283e service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:35.172350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725ddd3-7371-49ff-82c3-ab10e1fc283e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:35.182245 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725ddd3-7371-49ff-82c3-ab10e1fc283e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:35.183939 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725ddd3-7371-49ff-82c3-ab10e1fc283e service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:35.198150 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1120/2257] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 12:44:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 12:44:40.227821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d581fd0-de92-4c23-b622-bba7ca658faa None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:40.229213 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6d581fd0-de92-4c23-b622-bba7ca658faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:40.312574 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d581fd0-de92-4c23-b622-bba7ca658faa None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:40.318443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae44e6-290d-461f-a8c6-0ec3f0d6817a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff152e22374407eb72caf648a763107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:40.319065 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae44e6-290d-461f-a8c6-0ec3f0d6817a service placement] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:40.319385 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae44e6-290d-461f-a8c6-0ec3f0d6817a service placement] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:40.319662 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae44e6-290d-461f-a8c6-0ec3f0d6817a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:40.321467 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a4ae44e6-290d-461f-a8c6-0ec3f0d6817a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:40.334666 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae44e6-290d-461f-a8c6-0ec3f0d6817a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:40.336364 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae44e6-290d-461f-a8c6-0ec3f0d6817a service placement] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:40.347024 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1138/2258] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:44:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:44:45.188855 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=6 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:44:50.342228 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=35 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:44:52.055370 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931dfe2c-a16c-4398-90f8-aecea3cb620c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:52.055921 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931dfe2c-a16c-4398-90f8-aecea3cb620c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:52.056194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931dfe2c-a16c-4398-90f8-aecea3cb620c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:52.078250 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-931dfe2c-a16c-4398-90f8-aecea3cb620c None None] MFA Rules not processed for user `e42059e9af9842548ea08dcd39ff0b6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:44:52.081095 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-931dfe2c-a16c-4398-90f8-aecea3cb620c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:52.093452 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1121/2259] 10.222.0.49 () {58 vars in 967 bytes} [Wed Aug 10 12:44:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:44:52.112502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae44e6-290d-461f-a8c6-0ec3f0d6817a service placement] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:52.125315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae44e6-290d-461f-a8c6-0ec3f0d6817a service placement] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:52.131789 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663779c6-d8dd-4b39-8573-9381c0ea770d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59395bfa505c4540b3957deee78eb698', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:52.132400 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663779c6-d8dd-4b39-8573-9381c0ea770d service swift] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:52.132688 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663779c6-d8dd-4b39-8573-9381c0ea770d service swift] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:52.133181 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663779c6-d8dd-4b39-8573-9381c0ea770d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:52.142422 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663779c6-d8dd-4b39-8573-9381c0ea770d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:52.144271 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663779c6-d8dd-4b39-8573-9381c0ea770d service swift] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:52.154098 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1139/2260] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 12:44:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 12:44:53.939335 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed0b9d-965f-4288-b137-c6fd3246a374 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:53.939832 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed0b9d-965f-4288-b137-c6fd3246a374 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:53.940003 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed0b9d-965f-4288-b137-c6fd3246a374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:53.962483 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-baed0b9d-965f-4288-b137-c6fd3246a374 None None] MFA Rules not processed for user `e42059e9af9842548ea08dcd39ff0b6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:44:53.963738 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-baed0b9d-965f-4288-b137-c6fd3246a374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:53.970925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1122/2261] 10.222.0.49 () {58 vars in 967 bytes} [Wed Aug 10 12:44:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:44:53.983597 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663779c6-d8dd-4b39-8573-9381c0ea770d service swift] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:53.994487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663779c6-d8dd-4b39-8573-9381c0ea770d service swift] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:53.999932 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b1c31-c184-42f5-923c-b68f15e31e77 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59395bfa505c4540b3957deee78eb698', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:54.000737 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b1c31-c184-42f5-923c-b68f15e31e77 service swift] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:54.001023 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b1c31-c184-42f5-923c-b68f15e31e77 service swift] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:54.001313 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b1c31-c184-42f5-923c-b68f15e31e77 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:54.010498 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8b1c31-c184-42f5-923c-b68f15e31e77 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:54.012255 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8b1c31-c184-42f5-923c-b68f15e31e77 service swift] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:54.021033 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1140/2262] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 12:44:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 12:44:57.179940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed0b9d-965f-4288-b137-c6fd3246a374 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:57.181354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-baed0b9d-965f-4288-b137-c6fd3246a374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:57.250680 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed0b9d-965f-4288-b137-c6fd3246a374 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:57.255870 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afdc6f-74af-46bf-ad74-873532f5d628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:57.256506 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afdc6f-74af-46bf-ad74-873532f5d628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:57.256788 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afdc6f-74af-46bf-ad74-873532f5d628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:57.257068 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afdc6f-74af-46bf-ad74-873532f5d628 admin admin] PATH_INFO: `/v3/users/00b19e56e11047e3b0f102efea439b7f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:57.258297 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7afdc6f-74af-46bf-ad74-873532f5d628 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00b19e56e11047e3b0f102efea439b7f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:57.260274 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7afdc6f-74af-46bf-ad74-873532f5d628 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:57.443379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e7afdc6f-74af-46bf-ad74-873532f5d628 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00b19e56e11047e3b0f102efea439b7f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:57.515954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e7afdc6f-74af-46bf-ad74-873532f5d628 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00b19e56e11047e3b0f102efea439b7f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:57.516609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e7afdc6f-74af-46bf-ad74-873532f5d628 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00b19e56e11047e3b0f102efea439b7f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:57.517249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e7afdc6f-74af-46bf-ad74-873532f5d628 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00b19e56e11047e3b0f102efea439b7f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:57.525613 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e7afdc6f-74af-46bf-ad74-873532f5d628 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00b19e56e11047e3b0f102efea439b7f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:57.543971 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1123/2263] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 12:44:57 2022] DELETE /identity/v3/users/00b19e56e11047e3b0f102efea439b7f => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:44:57.798207 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b1c31-c184-42f5-923c-b68f15e31e77 service swift] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:57.809701 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b1c31-c184-42f5-923c-b68f15e31e77 service swift] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:57.877649 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7952e-bf30-4774-bdf9-e545924be3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:57.878282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7952e-bf30-4774-bdf9-e545924be3f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:57.878564 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7952e-bf30-4774-bdf9-e545924be3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:57.878839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7952e-bf30-4774-bdf9-e545924be3f8 admin admin] PATH_INFO: `/v3/projects/ca783a603c484aad86d05156d7e9142c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:57.881057 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7952e-bf30-4774-bdf9-e545924be3f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca783a603c484aad86d05156d7e9142c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:57.883110 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7952e-bf30-4774-bdf9-e545924be3f8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:57.986177 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e6f7952e-bf30-4774-bdf9-e545924be3f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca783a603c484aad86d05156d7e9142c', 'request_id': 'req-e6f7952e-bf30-4774-bdf9-e545924be3f8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:57.992847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e6f7952e-bf30-4774-bdf9-e545924be3f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca783a603c484aad86d05156d7e9142c', 'request_id': 'req-e6f7952e-bf30-4774-bdf9-e545924be3f8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:58.059908 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e6f7952e-bf30-4774-bdf9-e545924be3f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca783a603c484aad86d05156d7e9142c', 'request_id': 'req-e6f7952e-bf30-4774-bdf9-e545924be3f8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:58.072923 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e6f7952e-bf30-4774-bdf9-e545924be3f8 admin admin] The token cache is being invalidate because project ca783a603c484aad86d05156d7e9142c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:44:58.073220 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e6f7952e-bf30-4774-bdf9-e545924be3f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:58.073876 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e6f7952e-bf30-4774-bdf9-e545924be3f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:58.075400 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1141/2264] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 12:44:57 2022] DELETE /identity/v3/projects/ca783a603c484aad86d05156d7e9142c => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:44:58.080189 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a1851-6310-49bf-b8b0-b9a70cb27309 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:58.080475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a1851-6310-49bf-b8b0-b9a70cb27309 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:58.080752 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a1851-6310-49bf-b8b0-b9a70cb27309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:58.099958 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4e3a1851-6310-49bf-b8b0-b9a70cb27309 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:44:58.101157 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4e3a1851-6310-49bf-b8b0-b9a70cb27309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:58.163935 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1124/2265] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:44:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:44:58.171686 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52186ea-c592-4856-bd77-887cdffd65e7 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:58.171981 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52186ea-c592-4856-bd77-887cdffd65e7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:58.172261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52186ea-c592-4856-bd77-887cdffd65e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:58.192361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f52186ea-c592-4856-bd77-887cdffd65e7 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:44:58.193579 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f52186ea-c592-4856-bd77-887cdffd65e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:58.201480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1142/2266] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:44:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:44:58.205595 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a1851-6310-49bf-b8b0-b9a70cb27309 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:58.215308 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a1851-6310-49bf-b8b0-b9a70cb27309 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:58.219859 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df520a30-7c67-40ca-bbff-1a96dd9f8941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:58.220456 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df520a30-7c67-40ca-bbff-1a96dd9f8941 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:58.220740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df520a30-7c67-40ca-bbff-1a96dd9f8941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:58.221016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df520a30-7c67-40ca-bbff-1a96dd9f8941 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:58.221659 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df520a30-7c67-40ca-bbff-1a96dd9f8941 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:44:58.221967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df520a30-7c67-40ca-bbff-1a96dd9f8941 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:58.224110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df520a30-7c67-40ca-bbff-1a96dd9f8941 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:58.225550 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:44:58.236004 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1125/2267] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 12:44:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:44:58.242518 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52186ea-c592-4856-bd77-887cdffd65e7 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:58.252044 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52186ea-c592-4856-bd77-887cdffd65e7 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:58.257960 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b123e7-b7c9-48b1-8860-b40efcd67b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:58.258575 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b123e7-b7c9-48b1-8860-b40efcd67b92 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:58.259001 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b123e7-b7c9-48b1-8860-b40efcd67b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:58.259314 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b123e7-b7c9-48b1-8860-b40efcd67b92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:58.259943 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b123e7-b7c9-48b1-8860-b40efcd67b92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:58.261886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b123e7-b7c9-48b1-8860-b40efcd67b92 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:58.380962 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1143/2268] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 12:44:58 2022] POST /identity/v3/projects => generated 391 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:44:58.385222 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df520a30-7c67-40ca-bbff-1a96dd9f8941 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:58.395094 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df520a30-7c67-40ca-bbff-1a96dd9f8941 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:58.451251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0cbb24-e493-46b1-8a78-f38f6a6fe5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:58.451844 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0cbb24-e493-46b1-8a78-f38f6a6fe5cd admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:58.452126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0cbb24-e493-46b1-8a78-f38f6a6fe5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:58.452400 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0cbb24-e493-46b1-8a78-f38f6a6fe5cd admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:58.453014 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0cbb24-e493-46b1-8a78-f38f6a6fe5cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:58.454954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0cbb24-e493-46b1-8a78-f38f6a6fe5cd admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:58.527553 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1126/2269] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 12:44:58 2022] POST /identity/v3/users => generated 296 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:44:58.531601 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b123e7-b7c9-48b1-8860-b40efcd67b92 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:58.542448 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b123e7-b7c9-48b1-8860-b40efcd67b92 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:58.547108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e911d3c-7857-46f5-b041-c1695e1b67fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:58.547734 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e911d3c-7857-46f5-b041-c1695e1b67fb admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:58.548014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e911d3c-7857-46f5-b041-c1695e1b67fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:58.548288 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e911d3c-7857-46f5-b041-c1695e1b67fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:58.549023 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e911d3c-7857-46f5-b041-c1695e1b67fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:58.550617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e911d3c-7857-46f5-b041-c1695e1b67fb admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:58.560408 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1144/2270] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:44:58.564423 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0cbb24-e493-46b1-8a78-f38f6a6fe5cd admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:58.574365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0cbb24-e493-46b1-8a78-f38f6a6fe5cd admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:58.578972 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191f8be-4518-48ca-b08b-61c69da45b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:58.579615 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191f8be-4518-48ca-b08b-61c69da45b9f admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:58.579891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191f8be-4518-48ca-b08b-61c69da45b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:58.580158 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191f8be-4518-48ca-b08b-61c69da45b9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:58.580787 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d191f8be-4518-48ca-b08b-61c69da45b9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:58.582865 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d191f8be-4518-48ca-b08b-61c69da45b9f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:58.593180 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1127/2271] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 12:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:44:58.597149 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e911d3c-7857-46f5-b041-c1695e1b67fb admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:58.607680 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e911d3c-7857-46f5-b041-c1695e1b67fb admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:58.612362 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66d782-ff6b-4f4a-8f0b-b0a3022b4141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:58.613255 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66d782-ff6b-4f4a-8f0b-b0a3022b4141 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:58.613557 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66d782-ff6b-4f4a-8f0b-b0a3022b4141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:58.614161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66d782-ff6b-4f4a-8f0b-b0a3022b4141 admin admin] PATH_INFO: `/v3/projects/ad28b1fa20ea4c3a8ee880b0bc5bfae4/users/0321f2587fce4dcaa329cac228e836d6/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:58.628095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff66d782-ff6b-4f4a-8f0b-b0a3022b4141 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad28b1fa20ea4c3a8ee880b0bc5bfae4, user_id=0321f2587fce4dcaa329cac228e836d6, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:58.630938 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff66d782-ff6b-4f4a-8f0b-b0a3022b4141 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:58.718977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:44:58.724739 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1145/2272] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 12:44:58 2022] PUT /identity/v3/projects/ad28b1fa20ea4c3a8ee880b0bc5bfae4/users/0321f2587fce4dcaa329cac228e836d6/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:44:58.734976 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703dcdd4-231d-413c-97b5-a63bdbd1e96f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:58.735305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703dcdd4-231d-413c-97b5-a63bdbd1e96f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:58.735583 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703dcdd4-231d-413c-97b5-a63bdbd1e96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:58.764349 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-703dcdd4-231d-413c-97b5-a63bdbd1e96f None None] MFA Rules not processed for user `0321f2587fce4dcaa329cac228e836d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:44:58.765554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-703dcdd4-231d-413c-97b5-a63bdbd1e96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:58.829080 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1128/2273] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:44:58 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:44:58.840271 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66d782-ff6b-4f4a-8f0b-b0a3022b4141 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:58.842083 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ff66d782-ff6b-4f4a-8f0b-b0a3022b4141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:58.900383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66d782-ff6b-4f4a-8f0b-b0a3022b4141 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:58.903743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cfea5e-d56a-4f20-bab6-d0eaf3411b2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:58.904340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfea5e-d56a-4f20-bab6-d0eaf3411b2e service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:58.904750 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfea5e-d56a-4f20-bab6-d0eaf3411b2e service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:58.904887 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfea5e-d56a-4f20-bab6-d0eaf3411b2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:58.913906 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cfea5e-d56a-4f20-bab6-d0eaf3411b2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:58.915392 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cfea5e-d56a-4f20-bab6-d0eaf3411b2e service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:58.924706 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1146/2274] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:44:58 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:45:08.826280 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=69,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:45:08.920717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=26 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:46:09.489844 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c933240a-e32e-4fea-8446-a631a26ce577 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:46:09.497440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c933240a-e32e-4fea-8446-a631a26ce577 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:46:09.497440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c933240a-e32e-4fea-8446-a631a26ce577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:46:09.511116 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c933240a-e32e-4fea-8446-a631a26ce577 None None] MFA Rules not processed for user `39cb7e0c86b3460dbd17c58dc13ef0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:46:09.512353 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c933240a-e32e-4fea-8446-a631a26ce577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:46:09.690992 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1129/2275] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 12:46:09 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:46:19.684200 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=25 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:46:40.909622 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cfea5e-d56a-4f20-bab6-d0eaf3411b2e service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:46:40.911015 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d9cfea5e-d56a-4f20-bab6-d0eaf3411b2e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:46:40.984143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cfea5e-d56a-4f20-bab6-d0eaf3411b2e service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:46:40.989868 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b7b26-0669-4ce6-b220-c39f55141cc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:46:40.990473 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b7b26-0669-4ce6-b220-c39f55141cc4 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:46:40.990752 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b7b26-0669-4ce6-b220-c39f55141cc4 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:46:40.991026 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b7b26-0669-4ce6-b220-c39f55141cc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:46:40.992796 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ff0b7b26-0669-4ce6-b220-c39f55141cc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:46:41.050253 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b7b26-0669-4ce6-b220-c39f55141cc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:46:41.051954 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b7b26-0669-4ce6-b220-c39f55141cc4 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:46:41.061968 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1147/2276] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:46:40 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:46:41.561114 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c933240a-e32e-4fea-8446-a631a26ce577 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:46:41.569993 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c933240a-e32e-4fea-8446-a631a26ce577 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:46:41.574071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a03aa8-4320-47b0-9eb7-eff3b8c94c56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:46:41.574666 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a03aa8-4320-47b0-9eb7-eff3b8c94c56 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:46:41.574941 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a03aa8-4320-47b0-9eb7-eff3b8c94c56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:46:41.575244 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a03aa8-4320-47b0-9eb7-eff3b8c94c56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:46:41.576966 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c5a03aa8-4320-47b0-9eb7-eff3b8c94c56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:46:41.635052 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a03aa8-4320-47b0-9eb7-eff3b8c94c56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:46:41.636527 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a03aa8-4320-47b0-9eb7-eff3b8c94c56 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:46:41.645669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1130/2277] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:46:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:46:42.958678 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b7b26-0669-4ce6-b220-c39f55141cc4 service nova] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:46:42.960053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ff0b7b26-0669-4ce6-b220-c39f55141cc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:46:42.975759 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b7b26-0669-4ce6-b220-c39f55141cc4 service nova] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:46:42.981583 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1463e5f-48c6-462e-b679-ac5fb1d67126 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:46:42.982174 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1463e5f-48c6-462e-b679-ac5fb1d67126 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:46:42.982455 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1463e5f-48c6-462e-b679-ac5fb1d67126 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:46:42.983244 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1463e5f-48c6-462e-b679-ac5fb1d67126 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:46:42.984557 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b1463e5f-48c6-462e-b679-ac5fb1d67126 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:46:43.046807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1463e5f-48c6-462e-b679-ac5fb1d67126 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:46:43.048444 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1463e5f-48c6-462e-b679-ac5fb1d67126 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:46:43.058194 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1148/2278] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:46:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:46:47.070365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a03aa8-4320-47b0-9eb7-eff3b8c94c56 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:46:47.079305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a03aa8-4320-47b0-9eb7-eff3b8c94c56 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:46:47.082932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d39f0-2587-4386-884b-c88e2e8c8443 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:46:47.083562 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d39f0-2587-4386-884b-c88e2e8c8443 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:46:47.083848 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d39f0-2587-4386-884b-c88e2e8c8443 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:46:47.084119 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d39f0-2587-4386-884b-c88e2e8c8443 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:46:47.085842 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-646d39f0-2587-4386-884b-c88e2e8c8443 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:46:47.097239 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646d39f0-2587-4386-884b-c88e2e8c8443 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:46:47.098645 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646d39f0-2587-4386-884b-c88e2e8c8443 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:46:47.109171 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1131/2279] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:46:47 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:46:47.214695 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1463e5f-48c6-462e-b679-ac5fb1d67126 service nova] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:46:47.224016 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1463e5f-48c6-462e-b679-ac5fb1d67126 service nova] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:46:47.227650 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202a27b-e376-44da-9541-7e2110a31bb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:46:47.229084 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202a27b-e376-44da-9541-7e2110a31bb6 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:46:47.229364 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202a27b-e376-44da-9541-7e2110a31bb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:46:47.229703 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202a27b-e376-44da-9541-7e2110a31bb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:46:47.231472 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b202a27b-e376-44da-9541-7e2110a31bb6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:46:47.244965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b202a27b-e376-44da-9541-7e2110a31bb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:46:47.246386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b202a27b-e376-44da-9541-7e2110a31bb6 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:46:47.257334 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1149/2280] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:46:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:46:57.105395 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=34 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:46:57.252462 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:47:34.508597 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d39f0-2587-4386-884b-c88e2e8c8443 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:47:34.513295 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-646d39f0-2587-4386-884b-c88e2e8c8443 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:47:34.568991 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d39f0-2587-4386-884b-c88e2e8c8443 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:47:34.573777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f581b35-0b58-45ff-b9cb-bab4f38e56f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:47:34.574378 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f581b35-0b58-45ff-b9cb-bab4f38e56f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:47:34.574653 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f581b35-0b58-45ff-b9cb-bab4f38e56f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:47:34.574921 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f581b35-0b58-45ff-b9cb-bab4f38e56f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:47:34.576646 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2f581b35-0b58-45ff-b9cb-bab4f38e56f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:47:34.589618 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f581b35-0b58-45ff-b9cb-bab4f38e56f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:47:34.591031 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f581b35-0b58-45ff-b9cb-bab4f38e56f9 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:47:34.598879 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1132/2281] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:47:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:47:44.595316 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=26 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:48:54.839748 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202a27b-e376-44da-9541-7e2110a31bb6 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:48:54.841163 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b202a27b-e376-44da-9541-7e2110a31bb6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:48:54.851783 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202a27b-e376-44da-9541-7e2110a31bb6 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:48:54.855613 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c6ba3-0a31-471d-bdfd-b64532d69e12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:48:54.856207 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c6ba3-0a31-471d-bdfd-b64532d69e12 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:48:54.856503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c6ba3-0a31-471d-bdfd-b64532d69e12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:48:54.856762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c6ba3-0a31-471d-bdfd-b64532d69e12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:48:54.858747 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b07c6ba3-0a31-471d-bdfd-b64532d69e12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:48:54.918286 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c6ba3-0a31-471d-bdfd-b64532d69e12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:48:54.919764 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c6ba3-0a31-471d-bdfd-b64532d69e12 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:48:54.928741 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1150/2282] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:48:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:49:04.925057 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=28 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:49:58.300985 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f581b35-0b58-45ff-b9cb-bab4f38e56f9 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:49:58.340898 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f581b35-0b58-45ff-b9cb-bab4f38e56f9 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:49:58.344876 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d697ec-6ceb-4286-b713-71563cc1956e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:49:58.350351 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d697ec-6ceb-4286-b713-71563cc1956e service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:49:58.350351 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d697ec-6ceb-4286-b713-71563cc1956e service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:49:58.350351 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d697ec-6ceb-4286-b713-71563cc1956e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:49:58.377306 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d697ec-6ceb-4286-b713-71563cc1956e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:49:58.378810 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d697ec-6ceb-4286-b713-71563cc1956e service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:49:58.387726 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1133/2283] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:49:58 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:50:08.383632 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=24 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:50:31.790061 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c6ba3-0a31-471d-bdfd-b64532d69e12 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:50:31.791611 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b07c6ba3-0a31-471d-bdfd-b64532d69e12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:50:31.920038 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c6ba3-0a31-471d-bdfd-b64532d69e12 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:50:31.925152 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe857f-8794-4cc7-944a-6ee30ac5542a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff152e22374407eb72caf648a763107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:50:31.925750 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe857f-8794-4cc7-944a-6ee30ac5542a service placement] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:50:31.926041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe857f-8794-4cc7-944a-6ee30ac5542a service placement] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:50:31.926305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe857f-8794-4cc7-944a-6ee30ac5542a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:50:31.928110 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c0fe857f-8794-4cc7-944a-6ee30ac5542a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:50:31.939722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fe857f-8794-4cc7-944a-6ee30ac5542a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:50:31.941287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fe857f-8794-4cc7-944a-6ee30ac5542a service placement] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:50:31.950832 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1151/2284] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:50:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:50:41.946822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=59 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:51:40.916363 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d697ec-6ceb-4286-b713-71563cc1956e service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:51:40.924739 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d697ec-6ceb-4286-b713-71563cc1956e service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:51:40.929211 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db388054-ad63-4f65-b6f6-c3897a244af2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:51:40.933213 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db388054-ad63-4f65-b6f6-c3897a244af2 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:51:40.933213 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db388054-ad63-4f65-b6f6-c3897a244af2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:51:40.933213 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db388054-ad63-4f65-b6f6-c3897a244af2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:51:40.938518 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db388054-ad63-4f65-b6f6-c3897a244af2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:51:40.940008 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db388054-ad63-4f65-b6f6-c3897a244af2 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:51:40.966959 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1134/2285] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:51:40 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:51:41.914517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe857f-8794-4cc7-944a-6ee30ac5542a service placement] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:51:41.943099 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe857f-8794-4cc7-944a-6ee30ac5542a service placement] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:51:41.949970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e74ab-6c31-4851-9fe2-80b116229d40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:51:41.950567 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e74ab-6c31-4851-9fe2-80b116229d40 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:51:41.950847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e74ab-6c31-4851-9fe2-80b116229d40 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:51:41.951123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e74ab-6c31-4851-9fe2-80b116229d40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:51:41.959685 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e74ab-6c31-4851-9fe2-80b116229d40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:51:41.961240 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e74ab-6c31-4851-9fe2-80b116229d40 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:51:41.970030 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1152/2286] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:51:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:51:42.924034 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db388054-ad63-4f65-b6f6-c3897a244af2 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:51:42.936495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db388054-ad63-4f65-b6f6-c3897a244af2 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:51:42.941927 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759cbfd-7381-4700-affc-26f7ac74dd6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:51:42.942527 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759cbfd-7381-4700-affc-26f7ac74dd6c service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:51:42.942790 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759cbfd-7381-4700-affc-26f7ac74dd6c service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:51:42.943059 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759cbfd-7381-4700-affc-26f7ac74dd6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:51:42.950825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759cbfd-7381-4700-affc-26f7ac74dd6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:51:42.952408 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759cbfd-7381-4700-affc-26f7ac74dd6c service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:51:42.980129 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1135/2287] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:51:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:51:47.043187 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e74ab-6c31-4851-9fe2-80b116229d40 service nova] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:51:47.054425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e74ab-6c31-4851-9fe2-80b116229d40 service nova] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:51:47.059951 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed944c9-fcde-4cc9-86e1-e345086cffa2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:51:47.060563 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed944c9-fcde-4cc9-86e1-e345086cffa2 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:51:47.060842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed944c9-fcde-4cc9-86e1-e345086cffa2 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:51:47.061107 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed944c9-fcde-4cc9-86e1-e345086cffa2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:51:47.069865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed944c9-fcde-4cc9-86e1-e345086cffa2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:51:47.071452 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed944c9-fcde-4cc9-86e1-e345086cffa2 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:51:47.098596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1153/2288] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:51:47 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:51:52.977303 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=26 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:51:57.096061 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=26 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:52:38.493817 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759cbfd-7381-4700-affc-26f7ac74dd6c service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:52:38.513323 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759cbfd-7381-4700-affc-26f7ac74dd6c service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:52:38.516915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7291c4e6-5f57-420b-b1db-d2b2a689af14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:52:38.517511 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291c4e6-5f57-420b-b1db-d2b2a689af14 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:52:38.517790 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291c4e6-5f57-420b-b1db-d2b2a689af14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:52:38.518059 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291c4e6-5f57-420b-b1db-d2b2a689af14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:52:38.525266 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7291c4e6-5f57-420b-b1db-d2b2a689af14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:52:38.526697 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7291c4e6-5f57-420b-b1db-d2b2a689af14 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:52:38.534414 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1136/2289] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:52:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:52:48.530885 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=10 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:53:55.806282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed944c9-fcde-4cc9-86e1-e345086cffa2 service nova] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:53:55.816399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed944c9-fcde-4cc9-86e1-e345086cffa2 service nova] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:53:55.821242 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1f728-3a87-4897-891d-4a1af282827e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:53:55.821917 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1f728-3a87-4897-891d-4a1af282827e service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:53:55.822194 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1f728-3a87-4897-891d-4a1af282827e service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:53:55.822464 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1f728-3a87-4897-891d-4a1af282827e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:53:55.831830 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c1f728-3a87-4897-891d-4a1af282827e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:53:55.833886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c1f728-3a87-4897-891d-4a1af282827e service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:53:55.865010 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1154/2290] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:53:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:54:05.862099 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=13 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:54:58.340112 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7291c4e6-5f57-420b-b1db-d2b2a689af14 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:54:58.350875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7291c4e6-5f57-420b-b1db-d2b2a689af14 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:54:58.355559 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5b9c1-e716-49ca-8163-9f14b2f8f701 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:54:58.356149 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5b9c1-e716-49ca-8163-9f14b2f8f701 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:54:58.356427 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5b9c1-e716-49ca-8163-9f14b2f8f701 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:54:58.356696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5b9c1-e716-49ca-8163-9f14b2f8f701 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:54:58.364943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da5b9c1-e716-49ca-8163-9f14b2f8f701 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:54:58.366375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da5b9c1-e716-49ca-8163-9f14b2f8f701 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:54:58.375009 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1137/2291] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:54:58 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:55:08.371114 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=6 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:55:38.784935 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1f728-3a87-4897-891d-4a1af282827e service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:55:38.786397 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-16c1f728-3a87-4897-891d-4a1af282827e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:55:38.801774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1f728-3a87-4897-891d-4a1af282827e service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:55:38.807907 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66844cb2-06f5-4391-b768-4058013fde63 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff152e22374407eb72caf648a763107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:55:38.808499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66844cb2-06f5-4391-b768-4058013fde63 service placement] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:55:38.808776 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66844cb2-06f5-4391-b768-4058013fde63 service placement] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:55:38.809048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66844cb2-06f5-4391-b768-4058013fde63 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:55:38.817456 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66844cb2-06f5-4391-b768-4058013fde63 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:55:38.819026 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66844cb2-06f5-4391-b768-4058013fde63 service placement] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:55:38.829113 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1155/2292] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:55:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:55:48.824810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=6 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:56:41.050729 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5b9c1-e716-49ca-8163-9f14b2f8f701 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:56:41.103895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5b9c1-e716-49ca-8163-9f14b2f8f701 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:56:41.107735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d92ed-742e-4ed3-a585-a55eb53753cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:56:41.108333 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d92ed-742e-4ed3-a585-a55eb53753cb service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:56:41.108611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d92ed-742e-4ed3-a585-a55eb53753cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:56:41.108881 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d92ed-742e-4ed3-a585-a55eb53753cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:56:41.110710 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-849d92ed-742e-4ed3-a585-a55eb53753cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:56:41.186071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d92ed-742e-4ed3-a585-a55eb53753cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:56:41.187558 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d92ed-742e-4ed3-a585-a55eb53753cb service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:56:41.200694 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1138/2293] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:56:41 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:56:42.205181 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66844cb2-06f5-4391-b768-4058013fde63 service placement] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:56:42.206729 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-66844cb2-06f5-4391-b768-4058013fde63 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:56:42.298231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66844cb2-06f5-4391-b768-4058013fde63 service placement] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:56:42.303684 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499943b-5089-4f8c-9342-6f250e8d083d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:56:42.304290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499943b-5089-4f8c-9342-6f250e8d083d service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:56:42.304564 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499943b-5089-4f8c-9342-6f250e8d083d service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:56:42.304837 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499943b-5089-4f8c-9342-6f250e8d083d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:56:42.306685 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7499943b-5089-4f8c-9342-6f250e8d083d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:56:42.388657 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7499943b-5089-4f8c-9342-6f250e8d083d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:56:42.390260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7499943b-5089-4f8c-9342-6f250e8d083d service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:56:42.408551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1156/2294] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:56:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:56:43.121551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d92ed-742e-4ed3-a585-a55eb53753cb service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:56:43.130061 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d92ed-742e-4ed3-a585-a55eb53753cb service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:56:43.133808 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393bf515-2535-482f-b108-1aff4c9547ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:56:43.134498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393bf515-2535-482f-b108-1aff4c9547ce service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:56:43.135364 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393bf515-2535-482f-b108-1aff4c9547ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:56:43.136238 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393bf515-2535-482f-b108-1aff4c9547ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:56:43.138097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-393bf515-2535-482f-b108-1aff4c9547ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:56:43.208754 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393bf515-2535-482f-b108-1aff4c9547ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:56:43.210207 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393bf515-2535-482f-b108-1aff4c9547ce service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:56:43.219799 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1139/2295] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:56:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:56:47.313276 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499943b-5089-4f8c-9342-6f250e8d083d service nova] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:56:47.323055 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499943b-5089-4f8c-9342-6f250e8d083d service nova] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:56:47.326769 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de05f3-51d8-477a-a692-2287ba09075d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:56:47.327417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de05f3-51d8-477a-a692-2287ba09075d service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:56:47.327700 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de05f3-51d8-477a-a692-2287ba09075d service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:56:47.327973 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de05f3-51d8-477a-a692-2287ba09075d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:56:47.329823 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-43de05f3-51d8-477a-a692-2287ba09075d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:56:47.386676 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:56:47.400386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de05f3-51d8-477a-a692-2287ba09075d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:56:47.401809 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de05f3-51d8-477a-a692-2287ba09075d service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:56:47.410658 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1157/2296] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:56:47 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:56:53.215619 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=86 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:56:57.428054 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=7 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:57:17.252805 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393bf515-2535-482f-b108-1aff4c9547ce service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:57:17.263144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393bf515-2535-482f-b108-1aff4c9547ce service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:57:17.266859 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394a5db-e689-4193-ae40-0bcd7a6a4ce2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:57:17.267499 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394a5db-e689-4193-ae40-0bcd7a6a4ce2 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:57:17.267778 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394a5db-e689-4193-ae40-0bcd7a6a4ce2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:57:17.268048 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394a5db-e689-4193-ae40-0bcd7a6a4ce2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:57:17.269932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1394a5db-e689-4193-ae40-0bcd7a6a4ce2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:57:17.285252 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394a5db-e689-4193-ae40-0bcd7a6a4ce2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:57:17.286681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394a5db-e689-4193-ae40-0bcd7a6a4ce2 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:57:17.298637 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1140/2297] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:57:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:57:18.479282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de05f3-51d8-477a-a692-2287ba09075d service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:57:18.488761 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de05f3-51d8-477a-a692-2287ba09075d service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:57:18.492369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c981b32-5ca7-4b2a-b683-a58b79cb2106 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:57:18.492963 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c981b32-5ca7-4b2a-b683-a58b79cb2106 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:57:18.493244 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c981b32-5ca7-4b2a-b683-a58b79cb2106 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:57:18.493537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c981b32-5ca7-4b2a-b683-a58b79cb2106 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:57:18.495282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7c981b32-5ca7-4b2a-b683-a58b79cb2106 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:57:18.508761 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c981b32-5ca7-4b2a-b683-a58b79cb2106 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:57:18.510159 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c981b32-5ca7-4b2a-b683-a58b79cb2106 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:57:18.521439 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1158/2298] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:57:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:57:27.293684 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=6 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:57:28.516792 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=6 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:57:41.156140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394a5db-e689-4193-ae40-0bcd7a6a4ce2 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:57:41.157724 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1394a5db-e689-4193-ae40-0bcd7a6a4ce2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:57:41.224631 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394a5db-e689-4193-ae40-0bcd7a6a4ce2 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:57:41.228696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffbb16c-e428-4569-ad78-662360f752a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:57:41.229665 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbb16c-e428-4569-ad78-662360f752a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:57:41.229665 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbb16c-e428-4569-ad78-662360f752a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:57:41.229999 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbb16c-e428-4569-ad78-662360f752a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:57:41.232141 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6ffbb16c-e428-4569-ad78-662360f752a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:57:41.241945 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffbb16c-e428-4569-ad78-662360f752a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:57:41.243405 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffbb16c-e428-4569-ad78-662360f752a1 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:57:41.251993 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1141/2299] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:57:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:57:51.248322 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=31 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:58:55.008062 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c981b32-5ca7-4b2a-b683-a58b79cb2106 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:58:55.010066 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7c981b32-5ca7-4b2a-b683-a58b79cb2106 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:58:55.023537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c981b32-5ca7-4b2a-b683-a58b79cb2106 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:58:55.029092 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6eb2e-f174-4091-b748-da874ee2a4ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:58:55.029696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6eb2e-f174-4091-b748-da874ee2a4ef service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:58:55.029977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6eb2e-f174-4091-b748-da874ee2a4ef service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:58:55.030252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6eb2e-f174-4091-b748-da874ee2a4ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:58:55.032769 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-16d6eb2e-f174-4091-b748-da874ee2a4ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:58:55.103954 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d6eb2e-f174-4091-b748-da874ee2a4ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:58:55.105543 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d6eb2e-f174-4091-b748-da874ee2a4ef service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:58:55.115373 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1159/2300] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:58:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:59:05.111758 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=33 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:59:59.265272 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffbb16c-e428-4569-ad78-662360f752a1 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:59:59.266836 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6ffbb16c-e428-4569-ad78-662360f752a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:59:59.310303 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffbb16c-e428-4569-ad78-662360f752a1 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:59:59.313898 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c3bd3-18ea-4e2c-8c92-0f2fafecee64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:59:59.314488 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c3bd3-18ea-4e2c-8c92-0f2fafecee64 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:59:59.314767 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c3bd3-18ea-4e2c-8c92-0f2fafecee64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:59:59.315037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c3bd3-18ea-4e2c-8c92-0f2fafecee64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:59:59.316990 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-726c3bd3-18ea-4e2c-8c92-0f2fafecee64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:59:59.395314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c3bd3-18ea-4e2c-8c92-0f2fafecee64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:59:59.396733 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c3bd3-18ea-4e2c-8c92-0f2fafecee64 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:59:59.414752 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1142/2301] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 12:59:59 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:00:09.412318 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=54 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:00:41.854916 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6eb2e-f174-4091-b748-da874ee2a4ef service nova] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:00:41.856461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-16d6eb2e-f174-4091-b748-da874ee2a4ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:00:41.992170 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6eb2e-f174-4091-b748-da874ee2a4ef service nova] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:00:41.997379 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6dd84-b5aa-4095-b5f0-0393b0c569c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff152e22374407eb72caf648a763107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:00:41.997981 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6dd84-b5aa-4095-b5f0-0393b0c569c3 service placement] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:00:41.998256 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6dd84-b5aa-4095-b5f0-0393b0c569c3 service placement] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:00:41.998525 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6dd84-b5aa-4095-b5f0-0393b0c569c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:00:42.000405 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c6c6dd84-b5aa-4095-b5f0-0393b0c569c3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:00:42.012992 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c6dd84-b5aa-4095-b5f0-0393b0c569c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:00:42.014569 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c6dd84-b5aa-4095-b5f0-0393b0c569c3 service placement] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:00:42.025881 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1160/2302] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:00:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:00:52.020927 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=59 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:01:41.967360 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c3bd3-18ea-4e2c-8c92-0f2fafecee64 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:01:41.968919 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-726c3bd3-18ea-4e2c-8c92-0f2fafecee64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:01:42.011886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6dd84-b5aa-4095-b5f0-0393b0c569c3 service placement] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:01:42.022567 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c3bd3-18ea-4e2c-8c92-0f2fafecee64 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:01:42.027346 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957061c-f995-4466-8064-8babc416a345 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:01:42.027958 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957061c-f995-4466-8064-8babc416a345 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:01:42.028236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957061c-f995-4466-8064-8babc416a345 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:01:42.028504 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957061c-f995-4466-8064-8babc416a345 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:01:42.037935 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957061c-f995-4466-8064-8babc416a345 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:01:42.039686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957061c-f995-4466-8064-8babc416a345 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:01:42.047828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6dd84-b5aa-4095-b5f0-0393b0c569c3 service placement] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:01:42.051352 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1143/2303] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:01:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:01:42.054198 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d765e-02be-46e2-aaa0-96ca21c17d45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:01:42.054914 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d765e-02be-46e2-aaa0-96ca21c17d45 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:01:42.055069 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d765e-02be-46e2-aaa0-96ca21c17d45 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:01:42.055365 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d765e-02be-46e2-aaa0-96ca21c17d45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:01:42.063883 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d765e-02be-46e2-aaa0-96ca21c17d45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:01:42.065464 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d765e-02be-46e2-aaa0-96ca21c17d45 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:01:42.102519 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1161/2304] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:01:42 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:01:43.014810 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957061c-f995-4466-8064-8babc416a345 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:01:43.027959 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957061c-f995-4466-8064-8babc416a345 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:01:43.033486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73150d41-9296-46e4-9f08-bf2302c05f0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:01:43.034005 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73150d41-9296-46e4-9f08-bf2302c05f0a service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:01:43.034382 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73150d41-9296-46e4-9f08-bf2302c05f0a service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:01:43.034540 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73150d41-9296-46e4-9f08-bf2302c05f0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:01:43.042867 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73150d41-9296-46e4-9f08-bf2302c05f0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:01:43.044495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73150d41-9296-46e4-9f08-bf2302c05f0a service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:01:43.073016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1144/2305] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:01:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:01:47.978781 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d765e-02be-46e2-aaa0-96ca21c17d45 service nova] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:01:47.990000 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d765e-02be-46e2-aaa0-96ca21c17d45 service nova] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:01:47.995752 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6930af27-9ef6-47e2-95eb-ab20072f108b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:01:47.996381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930af27-9ef6-47e2-95eb-ab20072f108b service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:01:47.996659 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930af27-9ef6-47e2-95eb-ab20072f108b service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:01:47.996926 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930af27-9ef6-47e2-95eb-ab20072f108b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:01:48.005141 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6930af27-9ef6-47e2-95eb-ab20072f108b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:01:48.007419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6930af27-9ef6-47e2-95eb-ab20072f108b service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:01:48.036454 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1162/2306] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:01:47 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:01:53.070329 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=28 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:01:58.033343 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=33 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:02:17.197036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73150d41-9296-46e4-9f08-bf2302c05f0a service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:02:17.206163 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73150d41-9296-46e4-9f08-bf2302c05f0a service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:02:17.209982 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db609799-e438-43ad-8e95-afb1488334c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:02:17.210585 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db609799-e438-43ad-8e95-afb1488334c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:02:17.210867 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db609799-e438-43ad-8e95-afb1488334c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:02:17.211152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db609799-e438-43ad-8e95-afb1488334c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:02:17.221590 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db609799-e438-43ad-8e95-afb1488334c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:02:17.223023 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db609799-e438-43ad-8e95-afb1488334c1 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:02:17.234985 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1145/2307] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:02:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:02:18.416319 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6930af27-9ef6-47e2-95eb-ab20072f108b service nova] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:02:18.425675 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6930af27-9ef6-47e2-95eb-ab20072f108b service nova] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:02:18.431110 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097fc31-6fd9-402d-a727-5803ab8ef6c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:02:18.431784 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097fc31-6fd9-402d-a727-5803ab8ef6c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:02:18.432059 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097fc31-6fd9-402d-a727-5803ab8ef6c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:02:18.432328 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097fc31-6fd9-402d-a727-5803ab8ef6c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:02:18.443095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6097fc31-6fd9-402d-a727-5803ab8ef6c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:02:18.445464 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6097fc31-6fd9-402d-a727-5803ab8ef6c7 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:02:18.456911 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1163/2308] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:02:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:02:27.230147 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=6 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:02:28.451683 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=6 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:02:50.501905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db609799-e438-43ad-8e95-afb1488334c1 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:02:50.520475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db609799-e438-43ad-8e95-afb1488334c1 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:02:50.524652 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114f021-7fd8-4e31-b378-433f1fa14b55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:02:50.525242 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114f021-7fd8-4e31-b378-433f1fa14b55 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:02:50.525520 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114f021-7fd8-4e31-b378-433f1fa14b55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:02:50.525794 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114f021-7fd8-4e31-b378-433f1fa14b55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:02:50.533401 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114f021-7fd8-4e31-b378-433f1fa14b55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:02:50.534816 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114f021-7fd8-4e31-b378-433f1fa14b55 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:02:50.543761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1146/2309] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:02:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:03:00.539161 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=10 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:03:18.953369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097fc31-6fd9-402d-a727-5803ab8ef6c7 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:03:18.968967 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097fc31-6fd9-402d-a727-5803ab8ef6c7 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:03:18.976619 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993d001-5b17-4b5f-b264-f4ee421198b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:03:18.977243 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993d001-5b17-4b5f-b264-f4ee421198b0 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:03:18.977692 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993d001-5b17-4b5f-b264-f4ee421198b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:03:18.977801 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993d001-5b17-4b5f-b264-f4ee421198b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:03:18.979879 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a993d001-5b17-4b5f-b264-f4ee421198b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:03:19.005673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993d001-5b17-4b5f-b264-f4ee421198b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:03:19.007413 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993d001-5b17-4b5f-b264-f4ee421198b0 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:03:19.019415 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1164/2310] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:03:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:03:29.015359 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=6 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:03:40.697053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc5b2a4-78f9-4640-96c8-85b02ed23739 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:03:40.698155 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc5b2a4-78f9-4640-96c8-85b02ed23739 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:03:40.698155 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc5b2a4-78f9-4640-96c8-85b02ed23739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:03:40.725066 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9fc5b2a4-78f9-4640-96c8-85b02ed23739 None None] MFA Rules not processed for user `70010d27b7f44bc699ea5ad8093fec3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:03:40.726266 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9fc5b2a4-78f9-4640-96c8-85b02ed23739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:03:40.733967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1147/2311] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:03:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:03:41.037143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993d001-5b17-4b5f-b264-f4ee421198b0 service nova] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:03:41.046878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993d001-5b17-4b5f-b264-f4ee421198b0 service nova] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:03:41.050832 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a9d8f-5133-4d94-843e-6bfbb59b6bf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:03:41.051496 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a9d8f-5133-4d94-843e-6bfbb59b6bf7 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:03:41.051776 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a9d8f-5133-4d94-843e-6bfbb59b6bf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:03:41.052048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a9d8f-5133-4d94-843e-6bfbb59b6bf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:03:41.053806 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f06a9d8f-5133-4d94-843e-6bfbb59b6bf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:03:41.063379 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a9d8f-5133-4d94-843e-6bfbb59b6bf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:03:41.064900 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a9d8f-5133-4d94-843e-6bfbb59b6bf7 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:03:41.072743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1165/2312] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:03:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:03:41.474353 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc5b2a4-78f9-4640-96c8-85b02ed23739 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:03:41.487854 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc5b2a4-78f9-4640-96c8-85b02ed23739 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:03:41.493464 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12d638-c870-4ad3-8aea-82cf4801dc2f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff152e22374407eb72caf648a763107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:03:41.494220 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12d638-c870-4ad3-8aea-82cf4801dc2f service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:03:41.494705 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12d638-c870-4ad3-8aea-82cf4801dc2f service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:03:41.495118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12d638-c870-4ad3-8aea-82cf4801dc2f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:03:41.497097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7d12d638-c870-4ad3-8aea-82cf4801dc2f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:03:41.509496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12d638-c870-4ad3-8aea-82cf4801dc2f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:03:41.511773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12d638-c870-4ad3-8aea-82cf4801dc2f service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:03:41.523046 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1148/2313] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:03:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:03:51.069248 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=6 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:03:51.518329 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=12 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:03:52.094302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a9d8f-5133-4d94-843e-6bfbb59b6bf7 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:03:52.107873 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a9d8f-5133-4d94-843e-6bfbb59b6bf7 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:03:52.113382 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e8453-a836-42c7-8642-19d803917b7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:03:52.113972 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e8453-a836-42c7-8642-19d803917b7e service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:03:52.114252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e8453-a836-42c7-8642-19d803917b7e service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:03:52.114526 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e8453-a836-42c7-8642-19d803917b7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:03:52.116274 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8b4e8453-a836-42c7-8642-19d803917b7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:03:52.128418 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4e8453-a836-42c7-8642-19d803917b7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:03:52.129969 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4e8453-a836-42c7-8642-19d803917b7e service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:03:52.141062 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1166/2314] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:03:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:04:02.136644 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=6 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:04:07.832429 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12d638-c870-4ad3-8aea-82cf4801dc2f service placement] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:04:07.844032 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12d638-c870-4ad3-8aea-82cf4801dc2f service placement] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:04:07.847596 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d899c-4527-4368-9c2a-231e73a56e80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:04:07.848322 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d899c-4527-4368-9c2a-231e73a56e80 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:04:07.848601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d899c-4527-4368-9c2a-231e73a56e80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:04:07.848870 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d899c-4527-4368-9c2a-231e73a56e80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:04:07.858385 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d899c-4527-4368-9c2a-231e73a56e80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:04:07.859841 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d899c-4527-4368-9c2a-231e73a56e80 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:04:07.888567 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1149/2315] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:04:07 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:04:13.943157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec008311-668d-4b81-aef4-ade6754ed225 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:04:13.943665 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec008311-668d-4b81-aef4-ade6754ed225 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:04:13.943796 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec008311-668d-4b81-aef4-ade6754ed225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:04:13.971293 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ec008311-668d-4b81-aef4-ade6754ed225 None None] MFA Rules not processed for user `570c2a77b862411083b9444f8292192b`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:04:13.972584 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ec008311-668d-4b81-aef4-ade6754ed225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:04:13.985872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1167/2316] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:04:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:04:17.885380 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=13 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:04:23.968838 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=6 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:05:19.197345 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d899c-4527-4368-9c2a-231e73a56e80 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:05:19.207420 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d899c-4527-4368-9c2a-231e73a56e80 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:05:19.211137 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e619c03-23f4-44f9-83d2-6db2622afdb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:05:19.211763 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e619c03-23f4-44f9-83d2-6db2622afdb3 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:05:19.212036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e619c03-23f4-44f9-83d2-6db2622afdb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:05:19.212303 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e619c03-23f4-44f9-83d2-6db2622afdb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:05:19.222080 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e619c03-23f4-44f9-83d2-6db2622afdb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:05:19.223559 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e619c03-23f4-44f9-83d2-6db2622afdb3 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:05:19.233701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1150/2317] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:05:19 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:05:20.322585 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec008311-668d-4b81-aef4-ade6754ed225 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:05:20.323929 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ec008311-668d-4b81-aef4-ade6754ed225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:05:20.439276 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec008311-668d-4b81-aef4-ade6754ed225 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:05:20.444638 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e2ff0-a572-403f-ac46-df650381050f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:05:20.445270 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e2ff0-a572-403f-ac46-df650381050f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:05:20.445553 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e2ff0-a572-403f-ac46-df650381050f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:05:20.445826 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e2ff0-a572-403f-ac46-df650381050f admin admin] PATH_INFO: `/v3/users/0321f2587fce4dcaa329cac228e836d6` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:05:20.446984 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e2ff0-a572-403f-ac46-df650381050f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0321f2587fce4dcaa329cac228e836d6)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:05:20.448976 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e2ff0-a572-403f-ac46-df650381050f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:05:20.640373 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bb0e2ff0-a572-403f-ac46-df650381050f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0321f2587fce4dcaa329cac228e836d6'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:05:20.647527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bb0e2ff0-a572-403f-ac46-df650381050f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0321f2587fce4dcaa329cac228e836d6'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:05:20.700437 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bb0e2ff0-a572-403f-ac46-df650381050f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0321f2587fce4dcaa329cac228e836d6'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:05:20.701283 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bb0e2ff0-a572-403f-ac46-df650381050f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0321f2587fce4dcaa329cac228e836d6'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:05:20.701921 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bb0e2ff0-a572-403f-ac46-df650381050f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0321f2587fce4dcaa329cac228e836d6'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:05:20.720222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1168/2318] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:05:20 2022] DELETE /identity/v3/users/0321f2587fce4dcaa329cac228e836d6 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:05:20.725064 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d58f4a-db15-4930-b15d-8cf4cf77d7eb None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:05:20.725546 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d58f4a-db15-4930-b15d-8cf4cf77d7eb None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:05:20.725655 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d58f4a-db15-4930-b15d-8cf4cf77d7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:05:20.754007 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-69d58f4a-db15-4930-b15d-8cf4cf77d7eb None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:05:20.758329 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-69d58f4a-db15-4930-b15d-8cf4cf77d7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:05:20.814031 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1151/2319] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:05:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:05:20.824265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e2ff0-a572-403f-ac46-df650381050f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:05:20.825725 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bb0e2ff0-a572-403f-ac46-df650381050f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:05:20.882588 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e2ff0-a572-403f-ac46-df650381050f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:05:20.886214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e34db-045c-45ff-8c4b-b351b6f790db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:05:20.886808 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e34db-045c-45ff-8c4b-b351b6f790db service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:05:20.887092 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e34db-045c-45ff-8c4b-b351b6f790db service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:05:20.887399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e34db-045c-45ff-8c4b-b351b6f790db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:05:20.893326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:05:20.896953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e34db-045c-45ff-8c4b-b351b6f790db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:05:20.901319 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e34db-045c-45ff-8c4b-b351b6f790db service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:05:20.908298 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1169/2320] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:05:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:05:21.314223 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d58f4a-db15-4930-b15d-8cf4cf77d7eb None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:05:21.318295 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-69d58f4a-db15-4930-b15d-8cf4cf77d7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:05:21.330778 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d58f4a-db15-4930-b15d-8cf4cf77d7eb None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:05:21.335540 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606887e0-ce4b-428e-8923-2a3c655e346d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:05:21.336717 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606887e0-ce4b-428e-8923-2a3c655e346d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:05:21.337165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606887e0-ce4b-428e-8923-2a3c655e346d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:05:21.337451 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606887e0-ce4b-428e-8923-2a3c655e346d admin admin] PATH_INFO: `/v3/projects/ad28b1fa20ea4c3a8ee880b0bc5bfae4` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:05:21.338678 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606887e0-ce4b-428e-8923-2a3c655e346d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad28b1fa20ea4c3a8ee880b0bc5bfae4)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:05:21.340756 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606887e0-ce4b-428e-8923-2a3c655e346d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:05:21.449722 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-606887e0-ce4b-428e-8923-2a3c655e346d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad28b1fa20ea4c3a8ee880b0bc5bfae4', 'request_id': 'req-606887e0-ce4b-428e-8923-2a3c655e346d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:05:21.456176 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-606887e0-ce4b-428e-8923-2a3c655e346d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad28b1fa20ea4c3a8ee880b0bc5bfae4', 'request_id': 'req-606887e0-ce4b-428e-8923-2a3c655e346d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:05:21.529475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-606887e0-ce4b-428e-8923-2a3c655e346d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad28b1fa20ea4c3a8ee880b0bc5bfae4', 'request_id': 'req-606887e0-ce4b-428e-8923-2a3c655e346d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:05:21.541152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-606887e0-ce4b-428e-8923-2a3c655e346d admin admin] The token cache is being invalidate because project ad28b1fa20ea4c3a8ee880b0bc5bfae4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:05:21.541445 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-606887e0-ce4b-428e-8923-2a3c655e346d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:05:21.542079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-606887e0-ce4b-428e-8923-2a3c655e346d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:05:21.543611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1152/2321] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:05:21 2022] DELETE /identity/v3/projects/ad28b1fa20ea4c3a8ee880b0bc5bfae4 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:05:21.549443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b555d2-7e29-42cc-a6a9-e3795ae97bbf None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:05:21.550476 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b555d2-7e29-42cc-a6a9-e3795ae97bbf None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:05:21.550476 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b555d2-7e29-42cc-a6a9-e3795ae97bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:05:21.599872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c5b555d2-7e29-42cc-a6a9-e3795ae97bbf None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:05:21.601016 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c5b555d2-7e29-42cc-a6a9-e3795ae97bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:05:21.658825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1170/2322] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:05:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:05:21.668339 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb957b8a-87ac-4edf-a60e-427a0b48c7a6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:05:21.682162 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb957b8a-87ac-4edf-a60e-427a0b48c7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:05:21.682162 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb957b8a-87ac-4edf-a60e-427a0b48c7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:05:21.688870 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-cb957b8a-87ac-4edf-a60e-427a0b48c7a6 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:05:21.690077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cb957b8a-87ac-4edf-a60e-427a0b48c7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:05:21.697806 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1153/2323] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:05:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:05:21.702611 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b555d2-7e29-42cc-a6a9-e3795ae97bbf None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:05:21.712284 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b555d2-7e29-42cc-a6a9-e3795ae97bbf None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:05:21.716778 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9addc-51da-45da-aa7d-ee18cb6607fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:05:21.717365 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9addc-51da-45da-aa7d-ee18cb6607fb admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:05:21.717642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9addc-51da-45da-aa7d-ee18cb6607fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:05:21.717913 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9addc-51da-45da-aa7d-ee18cb6607fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:05:21.718551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f9addc-51da-45da-aa7d-ee18cb6607fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:05:21.718852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f9addc-51da-45da-aa7d-ee18cb6607fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:05:21.720547 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f9addc-51da-45da-aa7d-ee18cb6607fb admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:05:21.732013 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1171/2324] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 13:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:05:21.736598 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb957b8a-87ac-4edf-a60e-427a0b48c7a6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:05:21.745778 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb957b8a-87ac-4edf-a60e-427a0b48c7a6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:05:21.750942 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeeb868-9af0-4649-bd35-51a7eb941da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:05:21.751569 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeeb868-9af0-4649-bd35-51a7eb941da2 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:05:21.751851 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeeb868-9af0-4649-bd35-51a7eb941da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:05:21.752128 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeeb868-9af0-4649-bd35-51a7eb941da2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:05:21.752757 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeeb868-9af0-4649-bd35-51a7eb941da2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:05:21.754634 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeeb868-9af0-4649-bd35-51a7eb941da2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:05:21.862787 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1154/2325] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:05:21 2022] POST /identity/v3/projects => generated 397 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:05:21.866908 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9addc-51da-45da-aa7d-ee18cb6607fb admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:05:21.884166 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9addc-51da-45da-aa7d-ee18cb6607fb admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:05:21.932883 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b53d5-9d4d-4655-9e9f-5a19075eb3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:05:21.933476 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b53d5-9d4d-4655-9e9f-5a19075eb3cb admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:05:21.933757 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b53d5-9d4d-4655-9e9f-5a19075eb3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:05:21.934031 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b53d5-9d4d-4655-9e9f-5a19075eb3cb admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:05:21.934650 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b53d5-9d4d-4655-9e9f-5a19075eb3cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:05:21.937386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b53d5-9d4d-4655-9e9f-5a19075eb3cb admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:05:22.004059 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1172/2326] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:05:21 2022] POST /identity/v3/users => generated 299 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:05:22.008175 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeeb868-9af0-4649-bd35-51a7eb941da2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:05:22.018642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeeb868-9af0-4649-bd35-51a7eb941da2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:05:22.023074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd8800-8dc4-41cb-9d09-c3143c407e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:05:22.023707 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd8800-8dc4-41cb-9d09-c3143c407e49 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:05:22.023982 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd8800-8dc4-41cb-9d09-c3143c407e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:05:22.024250 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd8800-8dc4-41cb-9d09-c3143c407e49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:05:22.025037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd8800-8dc4-41cb-9d09-c3143c407e49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:05:22.026600 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd8800-8dc4-41cb-9d09-c3143c407e49 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:05:22.036620 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1155/2327] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:05:22.040407 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b53d5-9d4d-4655-9e9f-5a19075eb3cb admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:05:22.050817 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b53d5-9d4d-4655-9e9f-5a19075eb3cb admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:05:22.055375 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a49d1e-26ae-4814-b456-c9aa4a6f7fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:05:22.055958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a49d1e-26ae-4814-b456-c9aa4a6f7fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:05:22.056237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a49d1e-26ae-4814-b456-c9aa4a6f7fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:05:22.056508 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a49d1e-26ae-4814-b456-c9aa4a6f7fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:05:22.057133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a49d1e-26ae-4814-b456-c9aa4a6f7fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:05:22.058689 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a49d1e-26ae-4814-b456-c9aa4a6f7fd8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:05:22.069522 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1173/2328] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:05:22.073703 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd8800-8dc4-41cb-9d09-c3143c407e49 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:05:22.083871 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd8800-8dc4-41cb-9d09-c3143c407e49 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:05:22.088371 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e968a-c016-4a04-b38f-18d7dcd5fb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:05:22.089010 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e968a-c016-4a04-b38f-18d7dcd5fb37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:05:22.089288 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e968a-c016-4a04-b38f-18d7dcd5fb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:05:22.089561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e968a-c016-4a04-b38f-18d7dcd5fb37 admin admin] PATH_INFO: `/v3/projects/3f1e08c38df2441f9e475993c941a9e0/users/3de67b2dfa7746d2a178044afed10166/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:05:22.101237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:05:22.104150 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e968a-c016-4a04-b38f-18d7dcd5fb37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f1e08c38df2441f9e475993c941a9e0, user_id=3de67b2dfa7746d2a178044afed10166, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:05:22.107726 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e968a-c016-4a04-b38f-18d7dcd5fb37 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:05:22.157733 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1156/2329] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:05:22 2022] PUT /identity/v3/projects/3f1e08c38df2441f9e475993c941a9e0/users/3de67b2dfa7746d2a178044afed10166/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:05:22.168469 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc8297-c26f-4e79-bbb0-03851b1f04a1 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:05:22.170484 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc8297-c26f-4e79-bbb0-03851b1f04a1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:05:22.170484 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc8297-c26f-4e79-bbb0-03851b1f04a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:05:22.197840 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-61fc8297-c26f-4e79-bbb0-03851b1f04a1 None None] MFA Rules not processed for user `3de67b2dfa7746d2a178044afed10166`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:05:22.199067 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-61fc8297-c26f-4e79-bbb0-03851b1f04a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:05:22.222876 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:05:22.263066 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1174/2330] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:05:22 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:05:32.197636 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=3,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:05:32.272033 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=16 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:05:50.806154 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e968a-c016-4a04-b38f-18d7dcd5fb37 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:05:50.808016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d81e968a-c016-4a04-b38f-18d7dcd5fb37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:05:50.875981 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e968a-c016-4a04-b38f-18d7dcd5fb37 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:05:50.880875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70169611-b243-4fed-8f1b-71c4f830d204 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff152e22374407eb72caf648a763107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:05:50.881471 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70169611-b243-4fed-8f1b-71c4f830d204 service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:05:50.881748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70169611-b243-4fed-8f1b-71c4f830d204 service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:05:50.882017 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70169611-b243-4fed-8f1b-71c4f830d204 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:05:50.883727 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-70169611-b243-4fed-8f1b-71c4f830d204 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:05:50.894785 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70169611-b243-4fed-8f1b-71c4f830d204 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:05:50.896391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70169611-b243-4fed-8f1b-71c4f830d204 service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:05:50.906485 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1157/2331] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:05:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:06:00.901808 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=32 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:08.101869 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc8297-c26f-4e79-bbb0-03851b1f04a1 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:08.103394 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-61fc8297-c26f-4e79-bbb0-03851b1f04a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:08.160866 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc8297-c26f-4e79-bbb0-03851b1f04a1 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:08.164488 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1e6cc-4de0-43cf-b9a8-945750e87971 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:08.165087 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1e6cc-4de0-43cf-b9a8-945750e87971 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:08.165365 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1e6cc-4de0-43cf-b9a8-945750e87971 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:08.165636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1e6cc-4de0-43cf-b9a8-945750e87971 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:08.173899 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1e6cc-4de0-43cf-b9a8-945750e87971 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:08.175300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1e6cc-4de0-43cf-b9a8-945750e87971 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:08.183927 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1175/2332] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:06:08 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:06:18.179989 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=26 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:06:42.141807 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70169611-b243-4fed-8f1b-71c4f830d204 service placement] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:06:42.143249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-70169611-b243-4fed-8f1b-71c4f830d204 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:42.214222 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70169611-b243-4fed-8f1b-71c4f830d204 service placement] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:06:42.219549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38109a66-26d2-41a2-9624-b90bd3c35779 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:06:42.220143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38109a66-26d2-41a2-9624-b90bd3c35779 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:06:42.220421 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38109a66-26d2-41a2-9624-b90bd3c35779 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:06:42.220692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38109a66-26d2-41a2-9624-b90bd3c35779 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:06:42.222581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-38109a66-26d2-41a2-9624-b90bd3c35779 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:06:42.280893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38109a66-26d2-41a2-9624-b90bd3c35779 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:06:42.282070 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38109a66-26d2-41a2-9624-b90bd3c35779 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:06:42.291152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1158/2333] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:06:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:06:52.286962 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=56 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:20.412748 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1e6cc-4de0-43cf-b9a8-945750e87971 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:20.414174 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b2c1e6cc-4de0-43cf-b9a8-945750e87971 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:20.425190 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1e6cc-4de0-43cf-b9a8-945750e87971 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:20.429554 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fdcd5-6ec4-44f8-b0bb-34a34d7736d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:20.430138 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fdcd5-6ec4-44f8-b0bb-34a34d7736d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:20.430416 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fdcd5-6ec4-44f8-b0bb-34a34d7736d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:20.430687 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fdcd5-6ec4-44f8-b0bb-34a34d7736d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:20.432371 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-601fdcd5-6ec4-44f8-b0bb-34a34d7736d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:20.502246 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601fdcd5-6ec4-44f8-b0bb-34a34d7736d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:20.503713 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601fdcd5-6ec4-44f8-b0bb-34a34d7736d8 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:20.513054 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1176/2334] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:07:20 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:07:20.556828 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38109a66-26d2-41a2-9624-b90bd3c35779 service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:20.566204 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38109a66-26d2-41a2-9624-b90bd3c35779 service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:20.570040 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809bfaeb-5e2d-4e49-af66-ecae9bc4ce83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:20.573443 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bfaeb-5e2d-4e49-af66-ecae9bc4ce83 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:20.573443 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bfaeb-5e2d-4e49-af66-ecae9bc4ce83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:20.573443 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bfaeb-5e2d-4e49-af66-ecae9bc4ce83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:20.573443 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-809bfaeb-5e2d-4e49-af66-ecae9bc4ce83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:20.644545 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809bfaeb-5e2d-4e49-af66-ecae9bc4ce83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:20.646525 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809bfaeb-5e2d-4e49-af66-ecae9bc4ce83 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:20.655708 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1159/2335] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:07:20 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:07:20.702446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fdcd5-6ec4-44f8-b0bb-34a34d7736d8 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:20.711834 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fdcd5-6ec4-44f8-b0bb-34a34d7736d8 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:20.715576 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb5aaa-15f7-4780-87e2-f2f0d3ec4a26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:20.716274 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb5aaa-15f7-4780-87e2-f2f0d3ec4a26 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:20.716555 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb5aaa-15f7-4780-87e2-f2f0d3ec4a26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:20.716825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb5aaa-15f7-4780-87e2-f2f0d3ec4a26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:20.718560 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d9cb5aaa-15f7-4780-87e2-f2f0d3ec4a26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:20.790345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cb5aaa-15f7-4780-87e2-f2f0d3ec4a26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:20.791806 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cb5aaa-15f7-4780-87e2-f2f0d3ec4a26 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:20.800590 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1177/2336] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:07:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:07:30.651389 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=33 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:30.796462 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=66 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:07:55.169572 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809bfaeb-5e2d-4e49-af66-ecae9bc4ce83 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:55.171001 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-809bfaeb-5e2d-4e49-af66-ecae9bc4ce83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:55.237833 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809bfaeb-5e2d-4e49-af66-ecae9bc4ce83 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:55.241376 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e0b5a-ff3a-4107-9750-f20028a3c60b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:55.241972 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e0b5a-ff3a-4107-9750-f20028a3c60b service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:55.242916 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e0b5a-ff3a-4107-9750-f20028a3c60b service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:55.243350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e0b5a-ff3a-4107-9750-f20028a3c60b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:55.245075 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6e9e0b5a-ff3a-4107-9750-f20028a3c60b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:55.254210 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e0b5a-ff3a-4107-9750-f20028a3c60b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:55.255654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e0b5a-ff3a-4107-9750-f20028a3c60b service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:55.264092 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1160/2337] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:07:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:07:55.637069 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb5aaa-15f7-4780-87e2-f2f0d3ec4a26 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:07:55.646626 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb5aaa-15f7-4780-87e2-f2f0d3ec4a26 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:07:55.650533 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba44b4-f3f3-408b-9296-1d4846f0ced6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:07:55.651143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba44b4-f3f3-408b-9296-1d4846f0ced6 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:07:55.651468 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba44b4-f3f3-408b-9296-1d4846f0ced6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:07:55.651747 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba44b4-f3f3-408b-9296-1d4846f0ced6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:07:55.653591 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3dba44b4-f3f3-408b-9296-1d4846f0ced6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:07:55.682544 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba44b4-f3f3-408b-9296-1d4846f0ced6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:07:55.684016 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba44b4-f3f3-408b-9296-1d4846f0ced6 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:07:55.696103 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1178/2338] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:07:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:08:05.260216 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=31 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:08:05.690325 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=11 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:09:19.388958 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2648349-71ce-4365-973a-5acc2a9e9e18 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:19.389616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2648349-71ce-4365-973a-5acc2a9e9e18 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:19.389616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2648349-71ce-4365-973a-5acc2a9e9e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:19.438708 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b2648349-71ce-4365-973a-5acc2a9e9e18 None None] MFA Rules not processed for user `c0ada1accf1742b393e8a12c60074b9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:09:19.439952 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b2648349-71ce-4365-973a-5acc2a9e9e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:09:19.455854 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1161/2339] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:09:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:09:19.755201 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba44b4-f3f3-408b-9296-1d4846f0ced6 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:19.763843 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba44b4-f3f3-408b-9296-1d4846f0ced6 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:19.767692 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9d0b8-1ccb-4fcc-93a9-696ffe31e45b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:19.768318 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9d0b8-1ccb-4fcc-93a9-696ffe31e45b service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:19.768599 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9d0b8-1ccb-4fcc-93a9-696ffe31e45b service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:19.768900 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9d0b8-1ccb-4fcc-93a9-696ffe31e45b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:19.770634 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-02b9d0b8-1ccb-4fcc-93a9-696ffe31e45b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:09:19.780811 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b9d0b8-1ccb-4fcc-93a9-696ffe31e45b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:09:19.782232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b9d0b8-1ccb-4fcc-93a9-696ffe31e45b service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:09:19.790155 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1179/2340] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:09:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:09:20.180736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2648349-71ce-4365-973a-5acc2a9e9e18 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:20.192938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2648349-71ce-4365-973a-5acc2a9e9e18 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:20.197860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80677a77-1225-4293-8905-5dcbe6d60455 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff152e22374407eb72caf648a763107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:20.201116 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80677a77-1225-4293-8905-5dcbe6d60455 service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:20.201116 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80677a77-1225-4293-8905-5dcbe6d60455 service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:20.201116 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80677a77-1225-4293-8905-5dcbe6d60455 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:20.201116 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-80677a77-1225-4293-8905-5dcbe6d60455 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:09:20.213776 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80677a77-1225-4293-8905-5dcbe6d60455 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:09:20.215392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80677a77-1225-4293-8905-5dcbe6d60455 service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:09:20.226194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1162/2341] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:09:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:09:23.637413 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9d0b8-1ccb-4fcc-93a9-696ffe31e45b service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:23.639311 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-02b9d0b8-1ccb-4fcc-93a9-696ffe31e45b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:09:23.656211 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9d0b8-1ccb-4fcc-93a9-696ffe31e45b service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:23.661907 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282da32-64b7-4e33-8f86-33ffabdd6700 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:23.662647 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282da32-64b7-4e33-8f86-33ffabdd6700 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:23.662998 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282da32-64b7-4e33-8f86-33ffabdd6700 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:23.663383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282da32-64b7-4e33-8f86-33ffabdd6700 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:23.665702 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e282da32-64b7-4e33-8f86-33ffabdd6700 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:09:23.681049 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282da32-64b7-4e33-8f86-33ffabdd6700 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:09:23.682937 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282da32-64b7-4e33-8f86-33ffabdd6700 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:09:23.695199 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1180/2342] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:09:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:09:27.822898 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80677a77-1225-4293-8905-5dcbe6d60455 service placement] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:09:27.835961 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80677a77-1225-4293-8905-5dcbe6d60455 service placement] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:09:27.841458 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd4d36-bdd2-4b31-ad67-9851e3c5d783 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:09:27.842053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd4d36-bdd2-4b31-ad67-9851e3c5d783 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:09:27.842404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd4d36-bdd2-4b31-ad67-9851e3c5d783 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:09:27.842684 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd4d36-bdd2-4b31-ad67-9851e3c5d783 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:09:27.845160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-51dd4d36-bdd2-4b31-ad67-9851e3c5d783 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:09:27.857162 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd4d36-bdd2-4b31-ad67-9851e3c5d783 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:09:27.858967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd4d36-bdd2-4b31-ad67-9851e3c5d783 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:09:27.869572 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1163/2343] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:09:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:09:33.690432 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=12 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:09:37.865179 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=31 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:09.314582 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282da32-64b7-4e33-8f86-33ffabdd6700 service nova] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:09.341481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282da32-64b7-4e33-8f86-33ffabdd6700 service nova] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:09.345141 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ef558-2bcf-4707-9b58-278b615e8765 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:09.345734 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ef558-2bcf-4707-9b58-278b615e8765 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:09.346014 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ef558-2bcf-4707-9b58-278b615e8765 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:09.346288 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ef558-2bcf-4707-9b58-278b615e8765 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:09.348035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-693ef558-2bcf-4707-9b58-278b615e8765 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:10:09.418265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693ef558-2bcf-4707-9b58-278b615e8765 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:09.419655 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693ef558-2bcf-4707-9b58-278b615e8765 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:09.428519 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1181/2344] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:10:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:10:19.424376 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=40 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:10:55.804865 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd4d36-bdd2-4b31-ad67-9851e3c5d783 service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:10:55.806425 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-51dd4d36-bdd2-4b31-ad67-9851e3c5d783 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:10:55.872815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd4d36-bdd2-4b31-ad67-9851e3c5d783 service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:10:55.877999 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d88773-eed3-4479-b066-b6f22ef63bd0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff152e22374407eb72caf648a763107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:10:55.878601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d88773-eed3-4479-b066-b6f22ef63bd0 service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:10:55.878884 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d88773-eed3-4479-b066-b6f22ef63bd0 service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:10:55.879159 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d88773-eed3-4479-b066-b6f22ef63bd0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:10:55.888452 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d88773-eed3-4479-b066-b6f22ef63bd0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:10:55.890067 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d88773-eed3-4479-b066-b6f22ef63bd0 service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:10:55.899778 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1164/2345] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:10:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:11:05.895618 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=28 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:12:09.948656 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ef558-2bcf-4707-9b58-278b615e8765 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:12:09.968496 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ef558-2bcf-4707-9b58-278b615e8765 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:12:09.972328 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524bbc24-4219-484c-9e7e-ee19d02a071d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:12:09.972950 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524bbc24-4219-484c-9e7e-ee19d02a071d service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:12:09.973214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524bbc24-4219-484c-9e7e-ee19d02a071d service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:12:09.973487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524bbc24-4219-484c-9e7e-ee19d02a071d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:12:09.982728 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524bbc24-4219-484c-9e7e-ee19d02a071d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:12:09.984467 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524bbc24-4219-484c-9e7e-ee19d02a071d service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:12:09.993251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1182/2346] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:12:09 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:12:10.154232 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d88773-eed3-4479-b066-b6f22ef63bd0 service placement] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:12:10.173912 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d88773-eed3-4479-b066-b6f22ef63bd0 service placement] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:12:10.177529 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f489f6cf-aa37-45f9-9cf3-e01d2ef2b7d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:12:10.178118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489f6cf-aa37-45f9-9cf3-e01d2ef2b7d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:12:10.178396 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489f6cf-aa37-45f9-9cf3-e01d2ef2b7d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:12:10.178674 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489f6cf-aa37-45f9-9cf3-e01d2ef2b7d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:12:10.187478 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f489f6cf-aa37-45f9-9cf3-e01d2ef2b7d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:12:10.188928 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f489f6cf-aa37-45f9-9cf3-e01d2ef2b7d6 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:12:10.198521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1165/2347] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:12:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:12:19.989168 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=11 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:12:20.193759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=11 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:13:01.479080 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524bbc24-4219-484c-9e7e-ee19d02a071d service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:13:01.498444 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524bbc24-4219-484c-9e7e-ee19d02a071d service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:13:01.501997 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88530f7f-b689-4d03-bd4a-3e4a6990c12e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:13:01.502623 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88530f7f-b689-4d03-bd4a-3e4a6990c12e service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:13:01.502903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88530f7f-b689-4d03-bd4a-3e4a6990c12e service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:13:01.503193 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88530f7f-b689-4d03-bd4a-3e4a6990c12e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:13:01.510224 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88530f7f-b689-4d03-bd4a-3e4a6990c12e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:13:01.511672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88530f7f-b689-4d03-bd4a-3e4a6990c12e service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:13:01.519620 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1183/2348] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:13:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:13:11.516133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=10 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:13:21.334223 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f489f6cf-aa37-45f9-9cf3-e01d2ef2b7d6 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:13:21.342630 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f489f6cf-aa37-45f9-9cf3-e01d2ef2b7d6 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:13:21.346566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da2d2d-6260-4f29-9832-95b9e4d36b84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:13:21.347151 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da2d2d-6260-4f29-9832-95b9e4d36b84 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:13:21.347452 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da2d2d-6260-4f29-9832-95b9e4d36b84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:13:21.347720 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da2d2d-6260-4f29-9832-95b9e4d36b84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:13:21.355997 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da2d2d-6260-4f29-9832-95b9e4d36b84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:13:21.357388 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da2d2d-6260-4f29-9832-95b9e4d36b84 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:13:21.384874 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1166/2349] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:13:21 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:13:31.382293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=13 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:14:11.298678 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88530f7f-b689-4d03-bd4a-3e4a6990c12e service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:14:11.308368 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88530f7f-b689-4d03-bd4a-3e4a6990c12e service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:14:11.312098 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a79522-2282-41de-bfb7-66c7f90e8c87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:14:11.312759 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a79522-2282-41de-bfb7-66c7f90e8c87 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:14:11.313125 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a79522-2282-41de-bfb7-66c7f90e8c87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:14:11.313464 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a79522-2282-41de-bfb7-66c7f90e8c87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:14:11.322883 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a79522-2282-41de-bfb7-66c7f90e8c87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:14:11.324375 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a79522-2282-41de-bfb7-66c7f90e8c87 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:14:11.353488 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1184/2350] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:14:11 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:14:11.401335 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da2d2d-6260-4f29-9832-95b9e4d36b84 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:14:11.409988 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da2d2d-6260-4f29-9832-95b9e4d36b84 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:14:11.413615 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24866f5b-b9d5-49c9-be44-89c98a999789 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:14:11.414278 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24866f5b-b9d5-49c9-be44-89c98a999789 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:14:11.414553 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24866f5b-b9d5-49c9-be44-89c98a999789 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:14:11.414805 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24866f5b-b9d5-49c9-be44-89c98a999789 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:14:11.423397 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24866f5b-b9d5-49c9-be44-89c98a999789 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:14:11.424813 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24866f5b-b9d5-49c9-be44-89c98a999789 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:14:11.452684 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1167/2351] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:14:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:14:21.350724 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=13 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:14:21.449895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=13 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:15:21.460198 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a79522-2282-41de-bfb7-66c7f90e8c87 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:15:21.468874 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a79522-2282-41de-bfb7-66c7f90e8c87 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:15:21.472694 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07d382-9165-402d-a094-00106a124c1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:15:21.473297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07d382-9165-402d-a094-00106a124c1d service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:15:21.473610 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07d382-9165-402d-a094-00106a124c1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:15:21.473886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07d382-9165-402d-a094-00106a124c1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:15:21.482352 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07d382-9165-402d-a094-00106a124c1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:15:21.483809 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07d382-9165-402d-a094-00106a124c1d service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:15:21.511837 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1185/2352] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:15:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:15:31.508569 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=13 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:15:59.824292 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24866f5b-b9d5-49c9-be44-89c98a999789 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:15:59.890299 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24866f5b-b9d5-49c9-be44-89c98a999789 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:15:59.895422 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5d165-1791-4f8c-9681-b77cb83b5750 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff152e22374407eb72caf648a763107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:15:59.896023 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5d165-1791-4f8c-9681-b77cb83b5750 service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:15:59.896303 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5d165-1791-4f8c-9681-b77cb83b5750 service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:15:59.896573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5d165-1791-4f8c-9681-b77cb83b5750 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:15:59.898419 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-55e5d165-1791-4f8c-9681-b77cb83b5750 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:15:59.919782 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e5d165-1791-4f8c-9681-b77cb83b5750 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:15:59.922998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e5d165-1791-4f8c-9681-b77cb83b5750 service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:15:59.931020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1168/2353] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:15:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:16:09.926904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=36 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:14.420388 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07d382-9165-402d-a094-00106a124c1d service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:14.430713 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07d382-9165-402d-a094-00106a124c1d service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:14.434742 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87cec30-2fc2-4748-a349-7610ef71d9db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:14.435388 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cec30-2fc2-4748-a349-7610ef71d9db service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:14.435672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cec30-2fc2-4748-a349-7610ef71d9db service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:14.435947 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cec30-2fc2-4748-a349-7610ef71d9db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:14.454348 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87cec30-2fc2-4748-a349-7610ef71d9db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:14.457977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87cec30-2fc2-4748-a349-7610ef71d9db service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:14.480421 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1186/2354] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:16:15.071399 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5d165-1791-4f8c-9681-b77cb83b5750 service placement] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:15.073000 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-55e5d165-1791-4f8c-9681-b77cb83b5750 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:15.177937 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5d165-1791-4f8c-9681-b77cb83b5750 service placement] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:15.183927 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076f03c-c58b-4e33-a107-8fa213244355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:15.184578 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076f03c-c58b-4e33-a107-8fa213244355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:15.184855 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076f03c-c58b-4e33-a107-8fa213244355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:15.185128 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076f03c-c58b-4e33-a107-8fa213244355 admin admin] PATH_INFO: `/v3/users/3de67b2dfa7746d2a178044afed10166` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:15.199877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c076f03c-c58b-4e33-a107-8fa213244355 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3de67b2dfa7746d2a178044afed10166)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:15.202762 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c076f03c-c58b-4e33-a107-8fa213244355 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:15.412772 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c076f03c-c58b-4e33-a107-8fa213244355 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3de67b2dfa7746d2a178044afed10166'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:15.519432 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c076f03c-c58b-4e33-a107-8fa213244355 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3de67b2dfa7746d2a178044afed10166'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:15.520088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c076f03c-c58b-4e33-a107-8fa213244355 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3de67b2dfa7746d2a178044afed10166'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:15.520779 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c076f03c-c58b-4e33-a107-8fa213244355 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3de67b2dfa7746d2a178044afed10166'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:15.529557 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c076f03c-c58b-4e33-a107-8fa213244355 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3de67b2dfa7746d2a178044afed10166'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:15.548595 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1169/2355] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:16:15 2022] DELETE /identity/v3/users/3de67b2dfa7746d2a178044afed10166 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:15.553500 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58f112-30b1-47b5-8929-a575a6a2053b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:15.553500 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58f112-30b1-47b5-8929-a575a6a2053b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:15.556733 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58f112-30b1-47b5-8929-a575a6a2053b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:15.584354 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7f58f112-30b1-47b5-8929-a575a6a2053b None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:15.585585 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7f58f112-30b1-47b5-8929-a575a6a2053b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:15.652776 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1187/2356] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:15.668968 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076f03c-c58b-4e33-a107-8fa213244355 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:15.672308 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c076f03c-c58b-4e33-a107-8fa213244355 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:15.721291 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:15.738308 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076f03c-c58b-4e33-a107-8fa213244355 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:15.742504 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c4cfa-6347-421d-9f16-8b3fd363d72d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:15.743831 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c4cfa-6347-421d-9f16-8b3fd363d72d service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:15.744119 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c4cfa-6347-421d-9f16-8b3fd363d72d service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:15.744392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c4cfa-6347-421d-9f16-8b3fd363d72d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:15.755525 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564c4cfa-6347-421d-9f16-8b3fd363d72d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:15.759535 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564c4cfa-6347-421d-9f16-8b3fd363d72d service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:15.771430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1170/2357] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:16:16.804857 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58f112-30b1-47b5-8929-a575a6a2053b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:16.806499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7f58f112-30b1-47b5-8929-a575a6a2053b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:16.819633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58f112-30b1-47b5-8929-a575a6a2053b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:16.824863 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9fd03d-2800-453a-814b-5663f27ff215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:16.826066 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fd03d-2800-453a-814b-5663f27ff215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:16.826967 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fd03d-2800-453a-814b-5663f27ff215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:16.827694 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fd03d-2800-453a-814b-5663f27ff215 admin admin] PATH_INFO: `/v3/projects/3f1e08c38df2441f9e475993c941a9e0` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:16.846472 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9fd03d-2800-453a-814b-5663f27ff215 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f1e08c38df2441f9e475993c941a9e0)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:16.848673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9fd03d-2800-453a-814b-5663f27ff215 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:17.036096 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8b9fd03d-2800-453a-814b-5663f27ff215 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f1e08c38df2441f9e475993c941a9e0', 'request_id': 'req-8b9fd03d-2800-453a-814b-5663f27ff215'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:17.043686 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8b9fd03d-2800-453a-814b-5663f27ff215 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f1e08c38df2441f9e475993c941a9e0', 'request_id': 'req-8b9fd03d-2800-453a-814b-5663f27ff215'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:17.096866 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8b9fd03d-2800-453a-814b-5663f27ff215 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f1e08c38df2441f9e475993c941a9e0', 'request_id': 'req-8b9fd03d-2800-453a-814b-5663f27ff215'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:17.118583 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8b9fd03d-2800-453a-814b-5663f27ff215 admin admin] The token cache is being invalidate because project 3f1e08c38df2441f9e475993c941a9e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:16:17.120133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8b9fd03d-2800-453a-814b-5663f27ff215 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:17.120133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8b9fd03d-2800-453a-814b-5663f27ff215 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:17.126083 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1188/2358] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:16:16 2022] DELETE /identity/v3/projects/3f1e08c38df2441f9e475993c941a9e0 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:17.128451 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca96b8-2807-42d8-a233-ef92a24b1789 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:17.128977 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca96b8-2807-42d8-a233-ef92a24b1789 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:17.134487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca96b8-2807-42d8-a233-ef92a24b1789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:17.206489 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3fca96b8-2807-42d8-a233-ef92a24b1789 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:17.208634 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3fca96b8-2807-42d8-a233-ef92a24b1789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:17.325812 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1171/2359] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:17.337037 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a86a5-eb0d-4c4f-a334-1ed06da6f0d6 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:17.338663 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a86a5-eb0d-4c4f-a334-1ed06da6f0d6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:17.338663 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a86a5-eb0d-4c4f-a334-1ed06da6f0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:17.385884 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-1b6a86a5-eb0d-4c4f-a334-1ed06da6f0d6 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:17.389503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1b6a86a5-eb0d-4c4f-a334-1ed06da6f0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:17.408000 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1189/2360] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:17.414713 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca96b8-2807-42d8-a233-ef92a24b1789 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:17.425512 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca96b8-2807-42d8-a233-ef92a24b1789 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:17.448534 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410fd453-bafa-4f37-aea1-6964f27d7902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:17.448948 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fd453-bafa-4f37-aea1-6964f27d7902 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:17.449299 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fd453-bafa-4f37-aea1-6964f27d7902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:17.451832 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fd453-bafa-4f37-aea1-6964f27d7902 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:17.452797 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410fd453-bafa-4f37-aea1-6964f27d7902 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:16:17.453685 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410fd453-bafa-4f37-aea1-6964f27d7902 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:17.457425 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410fd453-bafa-4f37-aea1-6964f27d7902 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:17.489968 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1172/2361] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 13:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:16:17.493133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a86a5-eb0d-4c4f-a334-1ed06da6f0d6 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:17.504624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a86a5-eb0d-4c4f-a334-1ed06da6f0d6 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:17.521102 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70871c7b-60f2-41f1-b3d3-c0da3802ab19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:17.522559 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70871c7b-60f2-41f1-b3d3-c0da3802ab19 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:17.522559 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70871c7b-60f2-41f1-b3d3-c0da3802ab19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:17.522559 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70871c7b-60f2-41f1-b3d3-c0da3802ab19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:17.522870 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70871c7b-60f2-41f1-b3d3-c0da3802ab19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:17.525339 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70871c7b-60f2-41f1-b3d3-c0da3802ab19 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:17.674405 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1190/2362] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:16:17 2022] POST /identity/v3/projects => generated 383 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:16:17.678515 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410fd453-bafa-4f37-aea1-6964f27d7902 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:17.708539 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410fd453-bafa-4f37-aea1-6964f27d7902 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:17.806033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a77c03-54a6-4208-a8f8-2e389acfe4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:17.808435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a77c03-54a6-4208-a8f8-2e389acfe4dd admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:17.809160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a77c03-54a6-4208-a8f8-2e389acfe4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:17.809444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a77c03-54a6-4208-a8f8-2e389acfe4dd admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:17.810142 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a77c03-54a6-4208-a8f8-2e389acfe4dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:17.820013 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a77c03-54a6-4208-a8f8-2e389acfe4dd admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:17.991425 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1173/2363] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:16:17 2022] POST /identity/v3/users => generated 292 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:16:17.996619 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70871c7b-60f2-41f1-b3d3-c0da3802ab19 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:18.003277 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:18.012996 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70871c7b-60f2-41f1-b3d3-c0da3802ab19 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:18.024746 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8af3bb-8603-4689-a4fb-3fac2b1b73ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:18.037571 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8af3bb-8603-4689-a4fb-3fac2b1b73ef admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:18.037571 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8af3bb-8603-4689-a4fb-3fac2b1b73ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:18.037571 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8af3bb-8603-4689-a4fb-3fac2b1b73ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:18.037571 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8af3bb-8603-4689-a4fb-3fac2b1b73ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:18.037571 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8af3bb-8603-4689-a4fb-3fac2b1b73ef admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:18.065652 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1191/2364] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:16:18.071825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a77c03-54a6-4208-a8f8-2e389acfe4dd admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:18.101003 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a77c03-54a6-4208-a8f8-2e389acfe4dd admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:18.128632 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b24fd4-b669-465f-9de7-e80d850a36d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:18.129515 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b24fd4-b669-465f-9de7-e80d850a36d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:18.129802 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b24fd4-b669-465f-9de7-e80d850a36d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:18.130074 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b24fd4-b669-465f-9de7-e80d850a36d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:18.130874 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b24fd4-b669-465f-9de7-e80d850a36d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:18.132631 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b24fd4-b669-465f-9de7-e80d850a36d6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:18.157973 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1174/2365] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:16:18.161053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8af3bb-8603-4689-a4fb-3fac2b1b73ef admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:18.178051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8af3bb-8603-4689-a4fb-3fac2b1b73ef admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:18.191661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2abd9-6a97-41ab-9a86-18d1b45d1d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:18.193617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2abd9-6a97-41ab-9a86-18d1b45d1d63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:18.194055 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2abd9-6a97-41ab-9a86-18d1b45d1d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:18.194502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2abd9-6a97-41ab-9a86-18d1b45d1d63 admin admin] PATH_INFO: `/v3/projects/bd9b653c3c794b7192e659f2557e7335/users/c025957730c04df5980279aaef8d0807/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:18.242795 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f2abd9-6a97-41ab-9a86-18d1b45d1d63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd9b653c3c794b7192e659f2557e7335, user_id=c025957730c04df5980279aaef8d0807, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:18.245762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f2abd9-6a97-41ab-9a86-18d1b45d1d63 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:18.276697 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b24fd4-b669-465f-9de7-e80d850a36d6 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:18.278472 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-47b24fd4-b669-465f-9de7-e80d850a36d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:18.331424 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:18.352857 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1192/2366] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:16:18 2022] PUT /identity/v3/projects/bd9b653c3c794b7192e659f2557e7335/users/c025957730c04df5980279aaef8d0807/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:18.358329 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8a511-066f-4f8f-9d39-5271fa96477a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:18.358831 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8a511-066f-4f8f-9d39-5271fa96477a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:18.359474 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8a511-066f-4f8f-9d39-5271fa96477a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:18.404602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-92c8a511-066f-4f8f-9d39-5271fa96477a None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:18.406950 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-92c8a511-066f-4f8f-9d39-5271fa96477a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:18.417861 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b24fd4-b669-465f-9de7-e80d850a36d6 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:18.500834 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1193/2367] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:16:18.515830 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8a511-066f-4f8f-9d39-5271fa96477a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:18.517506 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-92c8a511-066f-4f8f-9d39-5271fa96477a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:18.590504 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49c52a-7cd6-432d-9880-b75e5bbae06a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:18.591148 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49c52a-7cd6-432d-9880-b75e5bbae06a service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:18.591477 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49c52a-7cd6-432d-9880-b75e5bbae06a service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:18.591751 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49c52a-7cd6-432d-9880-b75e5bbae06a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:18.594491 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-da49c52a-7cd6-432d-9880-b75e5bbae06a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:18.617651 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49c52a-7cd6-432d-9880-b75e5bbae06a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:18.621716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49c52a-7cd6-432d-9880-b75e5bbae06a service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:18.638327 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8a511-066f-4f8f-9d39-5271fa96477a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:18.651075 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4f725-8615-479d-aec3-5973d6a111a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:18.651789 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4f725-8615-479d-aec3-5973d6a111a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:18.652077 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4f725-8615-479d-aec3-5973d6a111a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:18.652351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4f725-8615-479d-aec3-5973d6a111a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:18.661267 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1175/2368] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:16:18.671682 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4f725-8615-479d-aec3-5973d6a111a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:18.673118 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4f725-8615-479d-aec3-5973d6a111a7 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:18.691494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1194/2369] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:16:18.747596 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49c52a-7cd6-432d-9880-b75e5bbae06a service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:18.769170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49c52a-7cd6-432d-9880-b75e5bbae06a service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:18.797311 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1799636f-2296-4a2a-b67a-837cb45f003c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:18.798460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799636f-2296-4a2a-b67a-837cb45f003c service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:18.798747 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799636f-2296-4a2a-b67a-837cb45f003c service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:18.799024 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799636f-2296-4a2a-b67a-837cb45f003c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:18.810771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1799636f-2296-4a2a-b67a-837cb45f003c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:18.859429 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1799636f-2296-4a2a-b67a-837cb45f003c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:18.861068 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1799636f-2296-4a2a-b67a-837cb45f003c service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:18.910158 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1176/2370] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:16:18.949820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4f725-8615-479d-aec3-5973d6a111a7 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:18.952215 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1aa4f725-8615-479d-aec3-5973d6a111a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:19.005978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4f725-8615-479d-aec3-5973d6a111a7 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:19.015110 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e560a2-87d4-4491-af6c-1d3628e7cdaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:19.016559 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e560a2-87d4-4491-af6c-1d3628e7cdaa service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:19.018517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e560a2-87d4-4491-af6c-1d3628e7cdaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:19.018821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e560a2-87d4-4491-af6c-1d3628e7cdaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:19.023503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a4e560a2-87d4-4491-af6c-1d3628e7cdaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:19.093371 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e560a2-87d4-4491-af6c-1d3628e7cdaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:19.094838 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e560a2-87d4-4491-af6c-1d3628e7cdaa service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:19.161632 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1195/2371] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:16:20.177301 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e9c37-bebd-4be6-a148-27d81fdc685c None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:20.178223 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e9c37-bebd-4be6-a148-27d81fdc685c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:20.178598 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e9c37-bebd-4be6-a148-27d81fdc685c None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:20.180140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1177/2372] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 10 13:16:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 13:16:20.224214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41266a05-82ea-4adb-8384-af4a78bda392 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:20.224849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41266a05-82ea-4adb-8384-af4a78bda392 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:20.224849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41266a05-82ea-4adb-8384-af4a78bda392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:20.300940 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-41266a05-82ea-4adb-8384-af4a78bda392 None None] MFA Rules not processed for user `3ff152e22374407eb72caf648a763107`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:20.302867 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-41266a05-82ea-4adb-8384-af4a78bda392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:20.339674 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:20.386738 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e9c37-bebd-4be6-a148-27d81fdc685c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:20.388674 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b62e9c37-bebd-4be6-a148-27d81fdc685c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:20.404215 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1196/2373] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 10 13:16:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:20.418954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e9c37-bebd-4be6-a148-27d81fdc685c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:20.420897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41266a05-82ea-4adb-8384-af4a78bda392 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:20.422458 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-41266a05-82ea-4adb-8384-af4a78bda392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:20.425373 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b03fc-d4a0-427b-b2c6-b35079832ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:20.426086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b03fc-d4a0-427b-b2c6-b35079832ac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:20.426593 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b03fc-d4a0-427b-b2c6-b35079832ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:20.427231 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b03fc-d4a0-427b-b2c6-b35079832ac8 admin admin] PATH_INFO: `/v3/users/d8e3d29dcfbc43a1aa2c26b676f5894c` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:20.429253 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b03fc-d4a0-427b-b2c6-b35079832ac8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8e3d29dcfbc43a1aa2c26b676f5894c)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:20.431302 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b03fc-d4a0-427b-b2c6-b35079832ac8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:20.436519 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41266a05-82ea-4adb-8384-af4a78bda392 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:20.441893 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516d69b1-622f-47de-b1eb-c078f2759765 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff152e22374407eb72caf648a763107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:20.442727 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d69b1-622f-47de-b1eb-c078f2759765 service placement] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:20.443010 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d69b1-622f-47de-b1eb-c078f2759765 service placement] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:20.443335 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d69b1-622f-47de-b1eb-c078f2759765 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:20.452384 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516d69b1-622f-47de-b1eb-c078f2759765 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:20.454052 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516d69b1-622f-47de-b1eb-c078f2759765 service placement] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:20.466793 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1197/2374] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:16:20.685424 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ed2b03fc-d4a0-427b-b2c6-b35079832ac8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8e3d29dcfbc43a1aa2c26b676f5894c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:20.765652 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ed2b03fc-d4a0-427b-b2c6-b35079832ac8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8e3d29dcfbc43a1aa2c26b676f5894c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:20.766653 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ed2b03fc-d4a0-427b-b2c6-b35079832ac8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8e3d29dcfbc43a1aa2c26b676f5894c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:20.767490 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ed2b03fc-d4a0-427b-b2c6-b35079832ac8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8e3d29dcfbc43a1aa2c26b676f5894c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:20.784335 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ed2b03fc-d4a0-427b-b2c6-b35079832ac8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8e3d29dcfbc43a1aa2c26b676f5894c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:20.815767 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1178/2375] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:16:20 2022] DELETE /identity/v3/users/d8e3d29dcfbc43a1aa2c26b676f5894c => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:21.147124 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516d69b1-622f-47de-b1eb-c078f2759765 service placement] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:21.169833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516d69b1-622f-47de-b1eb-c078f2759765 service placement] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:21.244139 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6dd09-a794-43ea-871f-7493a85c89c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:21.245942 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6dd09-a794-43ea-871f-7493a85c89c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:21.245942 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6dd09-a794-43ea-871f-7493a85c89c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:21.245942 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6dd09-a794-43ea-871f-7493a85c89c2 admin admin] PATH_INFO: `/v3/projects/4f4fa99fc6514cceb31af25c94623178` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:21.246762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6dd09-a794-43ea-871f-7493a85c89c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f4fa99fc6514cceb31af25c94623178)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:21.249690 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6dd09-a794-43ea-871f-7493a85c89c2 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:21.403064 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-15d6dd09-a794-43ea-871f-7493a85c89c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f4fa99fc6514cceb31af25c94623178', 'request_id': 'req-15d6dd09-a794-43ea-871f-7493a85c89c2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:21.411331 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-15d6dd09-a794-43ea-871f-7493a85c89c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f4fa99fc6514cceb31af25c94623178', 'request_id': 'req-15d6dd09-a794-43ea-871f-7493a85c89c2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:21.485083 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-15d6dd09-a794-43ea-871f-7493a85c89c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f4fa99fc6514cceb31af25c94623178', 'request_id': 'req-15d6dd09-a794-43ea-871f-7493a85c89c2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:21.508738 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-15d6dd09-a794-43ea-871f-7493a85c89c2 admin admin] The token cache is being invalidate because project 4f4fa99fc6514cceb31af25c94623178 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:16:21.509473 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-15d6dd09-a794-43ea-871f-7493a85c89c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:21.510293 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-15d6dd09-a794-43ea-871f-7493a85c89c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:21.528326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1198/2376] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:16:21 2022] DELETE /identity/v3/projects/4f4fa99fc6514cceb31af25c94623178 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:23.790407 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b03fc-d4a0-427b-b2c6-b35079832ac8 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:23.800478 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ed2b03fc-d4a0-427b-b2c6-b35079832ac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:23.805948 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:23.861236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b03fc-d4a0-427b-b2c6-b35079832ac8 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:23.930239 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c43de3-58a7-44e9-b288-9d127e8ad7b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:23.931527 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c43de3-58a7-44e9-b288-9d127e8ad7b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:23.931826 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c43de3-58a7-44e9-b288-9d127e8ad7b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:23.932290 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c43de3-58a7-44e9-b288-9d127e8ad7b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:23.934555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f1c43de3-58a7-44e9-b288-9d127e8ad7b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:24.032055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c43de3-58a7-44e9-b288-9d127e8ad7b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:24.035415 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c43de3-58a7-44e9-b288-9d127e8ad7b2 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:24.047993 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1179/2377] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:16:25.042942 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6dd09-a794-43ea-871f-7493a85c89c2 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:25.047573 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-15d6dd09-a794-43ea-871f-7493a85c89c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:25.062225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6dd09-a794-43ea-871f-7493a85c89c2 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:25.066229 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5aaa6-a0af-48d3-a866-90ff75c85bf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:25.067068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5aaa6-a0af-48d3-a866-90ff75c85bf3 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:25.067450 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5aaa6-a0af-48d3-a866-90ff75c85bf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:25.067789 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5aaa6-a0af-48d3-a866-90ff75c85bf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:25.069748 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-40c5aaa6-a0af-48d3-a866-90ff75c85bf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:25.084311 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c5aaa6-a0af-48d3-a866-90ff75c85bf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:25.089790 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c5aaa6-a0af-48d3-a866-90ff75c85bf3 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:25.098917 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1199/2378] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:16:26.672747 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b21553-bbc1-4ae4-9bee-6267c88a680c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:26.673265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b21553-bbc1-4ae4-9bee-6267c88a680c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:26.673265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b21553-bbc1-4ae4-9bee-6267c88a680c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:26.710644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d3b21553-bbc1-4ae4-9bee-6267c88a680c None None] MFA Rules not processed for user `570c2a77b862411083b9444f8292192b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:26.712168 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d3b21553-bbc1-4ae4-9bee-6267c88a680c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:26.839823 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1180/2379] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:16:26.970341 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140a676-4df5-4c36-b379-c1cadae93793 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:26.970872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140a676-4df5-4c36-b379-c1cadae93793 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:26.970872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140a676-4df5-4c36-b379-c1cadae93793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:27.004259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9140a676-4df5-4c36-b379-c1cadae93793 None None] MFA Rules not processed for user `c025957730c04df5980279aaef8d0807`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:27.005601 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9140a676-4df5-4c36-b379-c1cadae93793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:27.067278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:27.147376 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1200/2380] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:27.148173 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b21553-bbc1-4ae4-9bee-6267c88a680c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:27.149769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d3b21553-bbc1-4ae4-9bee-6267c88a680c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:27.169280 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b21553-bbc1-4ae4-9bee-6267c88a680c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:27.174137 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac9780-ae8b-44cc-9f77-c103ef78ff1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:27.174782 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac9780-ae8b-44cc-9f77-c103ef78ff1d admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:27.175088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac9780-ae8b-44cc-9f77-c103ef78ff1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:27.175396 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac9780-ae8b-44cc-9f77-c103ef78ff1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:27.176027 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac9780-ae8b-44cc-9f77-c103ef78ff1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:27.178535 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac9780-ae8b-44cc-9f77-c103ef78ff1d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:27.188910 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:27.273708 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140a676-4df5-4c36-b379-c1cadae93793 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:27.279092 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1181/2381] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:16:27 2022] POST /identity/v3/projects => generated 381 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:16:27.296701 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140a676-4df5-4c36-b379-c1cadae93793 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:27.430797 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee0ee3-0c86-4ecf-a108-5788509fe015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:27.432479 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee0ee3-0c86-4ecf-a108-5788509fe015 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:27.434517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee0ee3-0c86-4ecf-a108-5788509fe015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:27.435515 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee0ee3-0c86-4ecf-a108-5788509fe015 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:27.438341 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee0ee3-0c86-4ecf-a108-5788509fe015 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:27.444713 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee0ee3-0c86-4ecf-a108-5788509fe015 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:27.572535 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1201/2382] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:16:27 2022] POST /identity/v3/users => generated 297 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:16:27.577391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac9780-ae8b-44cc-9f77-c103ef78ff1d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:27.589474 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac9780-ae8b-44cc-9f77-c103ef78ff1d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:27.606570 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64847d-fe88-40d9-b65c-035a09cf9ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:27.607257 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64847d-fe88-40d9-b65c-035a09cf9ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:27.608460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64847d-fe88-40d9-b65c-035a09cf9ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:27.608830 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64847d-fe88-40d9-b65c-035a09cf9ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:27.610786 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f64847d-fe88-40d9-b65c-035a09cf9ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:27.613951 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f64847d-fe88-40d9-b65c-035a09cf9ca3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:27.647101 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1182/2383] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:16:27.647648 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee0ee3-0c86-4ecf-a108-5788509fe015 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:27.669518 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee0ee3-0c86-4ecf-a108-5788509fe015 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:27.676515 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df86f151-3fbd-4af4-a416-4807c79b25d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:27.678019 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86f151-3fbd-4af4-a416-4807c79b25d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:27.678383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86f151-3fbd-4af4-a416-4807c79b25d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:27.678792 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86f151-3fbd-4af4-a416-4807c79b25d5 admin admin] PATH_INFO: `/v3/projects/622cbf961ce84556945747e21c173eca/users/955679b46cd942608b7e13d09410ea01/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:27.703783 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df86f151-3fbd-4af4-a416-4807c79b25d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=622cbf961ce84556945747e21c173eca, user_id=955679b46cd942608b7e13d09410ea01, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:27.707849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df86f151-3fbd-4af4-a416-4807c79b25d5 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:27.782366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1202/2384] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:16:27 2022] PUT /identity/v3/projects/622cbf961ce84556945747e21c173eca/users/955679b46cd942608b7e13d09410ea01/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:16:27.786684 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64847d-fe88-40d9-b65c-035a09cf9ca3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:27.798736 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df86f151-3fbd-4af4-a416-4807c79b25d5 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:27.803659 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64847d-fe88-40d9-b65c-035a09cf9ca3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:27.805825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-df86f151-3fbd-4af4-a416-4807c79b25d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:27.924690 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df86f151-3fbd-4af4-a416-4807c79b25d5 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:27.930039 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da78ee-08f3-4e5c-87ea-18dba67a35be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:27.930814 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da78ee-08f3-4e5c-87ea-18dba67a35be service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:27.931623 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da78ee-08f3-4e5c-87ea-18dba67a35be service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:27.931993 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da78ee-08f3-4e5c-87ea-18dba67a35be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:27.934833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-72da78ee-08f3-4e5c-87ea-18dba67a35be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:27.948346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da78ee-08f3-4e5c-87ea-18dba67a35be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:27.951096 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47662dc-19d1-469c-b154-02afdb0ce65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:27.952236 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da78ee-08f3-4e5c-87ea-18dba67a35be service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:27.952539 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47662dc-19d1-469c-b154-02afdb0ce65d admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:27.952837 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47662dc-19d1-469c-b154-02afdb0ce65d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:27.953112 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47662dc-19d1-469c-b154-02afdb0ce65d admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:27.953763 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47662dc-19d1-469c-b154-02afdb0ce65d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:27.955391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47662dc-19d1-469c-b154-02afdb0ce65d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:27.962733 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1203/2385] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:16:27.983501 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1183/2386] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:16:27.989423 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da78ee-08f3-4e5c-87ea-18dba67a35be service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:28.006362 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da78ee-08f3-4e5c-87ea-18dba67a35be service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:28.015088 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494008a-9f41-461b-935b-ec816e7ac8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:28.019833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494008a-9f41-461b-935b-ec816e7ac8a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:28.020165 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494008a-9f41-461b-935b-ec816e7ac8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:28.021097 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494008a-9f41-461b-935b-ec816e7ac8a8 admin admin] PATH_INFO: `/v3/projects/622cbf961ce84556945747e21c173eca/users/955679b46cd942608b7e13d09410ea01/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:28.028310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a494008a-9f41-461b-935b-ec816e7ac8a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=622cbf961ce84556945747e21c173eca, user_id=955679b46cd942608b7e13d09410ea01, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:28.031067 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a494008a-9f41-461b-935b-ec816e7ac8a8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:28.059672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1204/2387] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:16:27 2022] PUT /identity/v3/projects/622cbf961ce84556945747e21c173eca/users/955679b46cd942608b7e13d09410ea01/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:30.389597 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47662dc-19d1-469c-b154-02afdb0ce65d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:30.391834 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b47662dc-19d1-469c-b154-02afdb0ce65d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:30.525239 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494008a-9f41-461b-935b-ec816e7ac8a8 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:30.527559 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a494008a-9f41-461b-935b-ec816e7ac8a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:30.564753 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47662dc-19d1-469c-b154-02afdb0ce65d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:30.599672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:30.694991 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494008a-9f41-461b-935b-ec816e7ac8a8 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:30.698027 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4095eb-5c76-4959-a9b3-42fc12e61264 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff152e22374407eb72caf648a763107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:30.699612 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4095eb-5c76-4959-a9b3-42fc12e61264 service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:30.700382 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4095eb-5c76-4959-a9b3-42fc12e61264 service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:30.706108 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4095eb-5c76-4959-a9b3-42fc12e61264 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:30.706108 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-db4095eb-5c76-4959-a9b3-42fc12e61264 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:30.709283 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f978110-1fbe-4299-b439-5fc8305c267e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff152e22374407eb72caf648a763107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:30.710197 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f978110-1fbe-4299-b439-5fc8305c267e service placement] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:30.710660 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f978110-1fbe-4299-b439-5fc8305c267e service placement] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:30.711111 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f978110-1fbe-4299-b439-5fc8305c267e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:30.719827 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2f978110-1fbe-4299-b439-5fc8305c267e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:30.733595 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4095eb-5c76-4959-a9b3-42fc12e61264 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:30.736012 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f978110-1fbe-4299-b439-5fc8305c267e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:30.737369 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4095eb-5c76-4959-a9b3-42fc12e61264 service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:30.738536 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f978110-1fbe-4299-b439-5fc8305c267e service placement] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:30.762504 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1184/2388] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:16:30.765391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4095eb-5c76-4959-a9b3-42fc12e61264 service placement] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:30.767866 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-db4095eb-5c76-4959-a9b3-42fc12e61264 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:30.768173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1205/2389] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:16:30.787821 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:30.929587 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4095eb-5c76-4959-a9b3-42fc12e61264 service placement] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:30.932585 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0487f9-8723-411b-886b-1887e8184cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:30.933291 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0487f9-8723-411b-886b-1887e8184cf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:30.933577 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0487f9-8723-411b-886b-1887e8184cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:30.933848 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0487f9-8723-411b-886b-1887e8184cf4 admin admin] PATH_INFO: `/v3/users/39cb7e0c86b3460dbd17c58dc13ef0d9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:30.935016 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0487f9-8723-411b-886b-1887e8184cf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39cb7e0c86b3460dbd17c58dc13ef0d9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:30.937372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0487f9-8723-411b-886b-1887e8184cf4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:31.229384 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1c0487f9-8723-411b-886b-1887e8184cf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39cb7e0c86b3460dbd17c58dc13ef0d9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:31.308135 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1c0487f9-8723-411b-886b-1887e8184cf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39cb7e0c86b3460dbd17c58dc13ef0d9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:31.308823 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1c0487f9-8723-411b-886b-1887e8184cf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39cb7e0c86b3460dbd17c58dc13ef0d9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:31.309497 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1c0487f9-8723-411b-886b-1887e8184cf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39cb7e0c86b3460dbd17c58dc13ef0d9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:31.323423 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1c0487f9-8723-411b-886b-1887e8184cf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39cb7e0c86b3460dbd17c58dc13ef0d9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:31.359884 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1185/2390] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:16:30 2022] DELETE /identity/v3/users/39cb7e0c86b3460dbd17c58dc13ef0d9 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:31.831890 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f978110-1fbe-4299-b439-5fc8305c267e service placement] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:31.856348 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f978110-1fbe-4299-b439-5fc8305c267e service placement] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:31.947787 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0487f9-8723-411b-886b-1887e8184cf4 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:31.949388 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1c0487f9-8723-411b-886b-1887e8184cf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:31.989745 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805de39-4716-40e6-bba2-946890e8c45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:31.989745 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805de39-4716-40e6-bba2-946890e8c45a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:31.989745 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805de39-4716-40e6-bba2-946890e8c45a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:31.989745 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805de39-4716-40e6-bba2-946890e8c45a admin admin] PATH_INFO: `/v3/projects/e41d5b8acbce44248a45678d1e4f6b71` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:31.990104 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805de39-4716-40e6-bba2-946890e8c45a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e41d5b8acbce44248a45678d1e4f6b71)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:31.992161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805de39-4716-40e6-bba2-946890e8c45a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:32.058604 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0487f9-8723-411b-886b-1887e8184cf4 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:32.069503 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f86d01a-6727-4d81-91d1-06c75ca5f995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:32.070257 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86d01a-6727-4d81-91d1-06c75ca5f995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:32.070572 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86d01a-6727-4d81-91d1-06c75ca5f995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:32.070855 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86d01a-6727-4d81-91d1-06c75ca5f995 admin admin] PATH_INFO: `/v3/users/70010d27b7f44bc699ea5ad8093fec3b` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:32.072543 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f86d01a-6727-4d81-91d1-06c75ca5f995 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70010d27b7f44bc699ea5ad8093fec3b)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:32.074525 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f86d01a-6727-4d81-91d1-06c75ca5f995 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:32.138694 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a805de39-4716-40e6-bba2-946890e8c45a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e41d5b8acbce44248a45678d1e4f6b71', 'request_id': 'req-a805de39-4716-40e6-bba2-946890e8c45a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:32.146175 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a805de39-4716-40e6-bba2-946890e8c45a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e41d5b8acbce44248a45678d1e4f6b71', 'request_id': 'req-a805de39-4716-40e6-bba2-946890e8c45a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:32.184176 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:32.260641 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a805de39-4716-40e6-bba2-946890e8c45a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e41d5b8acbce44248a45678d1e4f6b71', 'request_id': 'req-a805de39-4716-40e6-bba2-946890e8c45a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:32.292497 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a805de39-4716-40e6-bba2-946890e8c45a admin admin] The token cache is being invalidate because project e41d5b8acbce44248a45678d1e4f6b71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:16:32.293375 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a805de39-4716-40e6-bba2-946890e8c45a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:32.297127 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a805de39-4716-40e6-bba2-946890e8c45a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:32.312212 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1206/2391] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:16:31 2022] DELETE /identity/v3/projects/e41d5b8acbce44248a45678d1e4f6b71 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:32.314636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f101e8-16b0-475c-a3c3-b8ee547c4f37 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:32.314922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f101e8-16b0-475c-a3c3-b8ee547c4f37 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:32.315223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f101e8-16b0-475c-a3c3-b8ee547c4f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:32.315511 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7f86d01a-6727-4d81-91d1-06c75ca5f995 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70010d27b7f44bc699ea5ad8093fec3b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:32.359555 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c4f101e8-16b0-475c-a3c3-b8ee547c4f37 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:32.360845 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c4f101e8-16b0-475c-a3c3-b8ee547c4f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:32.387747 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7f86d01a-6727-4d81-91d1-06c75ca5f995 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70010d27b7f44bc699ea5ad8093fec3b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:32.389362 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7f86d01a-6727-4d81-91d1-06c75ca5f995 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70010d27b7f44bc699ea5ad8093fec3b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:32.390400 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7f86d01a-6727-4d81-91d1-06c75ca5f995 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70010d27b7f44bc699ea5ad8093fec3b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:32.402075 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7f86d01a-6727-4d81-91d1-06c75ca5f995 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70010d27b7f44bc699ea5ad8093fec3b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:32.407360 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:32.428482 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1186/2392] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:16:31 2022] DELETE /identity/v3/users/70010d27b7f44bc699ea5ad8093fec3b => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:16:32.438067 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1207/2393] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:32.447817 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b799123-b2e1-4101-bc1a-21ea20cbe60a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:32.447817 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b799123-b2e1-4101-bc1a-21ea20cbe60a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:32.447817 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b799123-b2e1-4101-bc1a-21ea20cbe60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:32.467531 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7b799123-b2e1-4101-bc1a-21ea20cbe60a None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:32.471152 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7b799123-b2e1-4101-bc1a-21ea20cbe60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:32.563285 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1187/2394] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:32.567033 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f101e8-16b0-475c-a3c3-b8ee547c4f37 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:32.577545 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f101e8-16b0-475c-a3c3-b8ee547c4f37 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:32.592863 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f1982-0677-4990-82ab-6ae784991cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:32.594707 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f1982-0677-4990-82ab-6ae784991cec admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:32.595206 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f1982-0677-4990-82ab-6ae784991cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:32.596335 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f1982-0677-4990-82ab-6ae784991cec admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:32.597251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f1982-0677-4990-82ab-6ae784991cec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:16:32.597801 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f1982-0677-4990-82ab-6ae784991cec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:32.600271 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f1982-0677-4990-82ab-6ae784991cec admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:32.624957 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1208/2395] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 13:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:16:32.629438 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b799123-b2e1-4101-bc1a-21ea20cbe60a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:32.651509 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b799123-b2e1-4101-bc1a-21ea20cbe60a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:32.662029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b91b3c-3502-469f-a3e4-8dec16e27570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:32.663080 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b91b3c-3502-469f-a3e4-8dec16e27570 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:32.663498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b91b3c-3502-469f-a3e4-8dec16e27570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:32.663870 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b91b3c-3502-469f-a3e4-8dec16e27570 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:32.664620 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b91b3c-3502-469f-a3e4-8dec16e27570 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:32.666881 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b91b3c-3502-469f-a3e4-8dec16e27570 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:32.811661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f1982-0677-4990-82ab-6ae784991cec admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:32.813524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ca2f1982-0677-4990-82ab-6ae784991cec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:32.843388 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1188/2396] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:16:32 2022] POST /identity/v3/projects => generated 373 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:16:32.847878 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b91b3c-3502-469f-a3e4-8dec16e27570 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:32.851022 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f1982-0677-4990-82ab-6ae784991cec admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:32.937963 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b91b3c-3502-469f-a3e4-8dec16e27570 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:32.944657 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8734f3d1-2c6b-4898-92e7-9eaf8f0fdd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:32.945312 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734f3d1-2c6b-4898-92e7-9eaf8f0fdd75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:32.945691 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734f3d1-2c6b-4898-92e7-9eaf8f0fdd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:32.946072 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734f3d1-2c6b-4898-92e7-9eaf8f0fdd75 admin admin] PATH_INFO: `/v3/projects/d6fe8b6c27e743eaad5fe6ffc8a40e81` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:32.947995 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356cbe8b-af3f-4d0c-bfbd-cccb02bd699c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:32.948447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8734f3d1-2c6b-4898-92e7-9eaf8f0fdd75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6fe8b6c27e743eaad5fe6ffc8a40e81)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:32.948649 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356cbe8b-af3f-4d0c-bfbd-cccb02bd699c admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:32.948903 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356cbe8b-af3f-4d0c-bfbd-cccb02bd699c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:32.949677 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356cbe8b-af3f-4d0c-bfbd-cccb02bd699c admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:32.949855 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356cbe8b-af3f-4d0c-bfbd-cccb02bd699c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:32.951028 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8734f3d1-2c6b-4898-92e7-9eaf8f0fdd75 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:32.952764 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356cbe8b-af3f-4d0c-bfbd-cccb02bd699c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:33.069099 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1189/2397] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:16:32 2022] POST /identity/v3/users => generated 287 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:16:33.073409 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356cbe8b-af3f-4d0c-bfbd-cccb02bd699c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:33.085317 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356cbe8b-af3f-4d0c-bfbd-cccb02bd699c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:33.086528 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8734f3d1-2c6b-4898-92e7-9eaf8f0fdd75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6fe8b6c27e743eaad5fe6ffc8a40e81', 'request_id': 'req-8734f3d1-2c6b-4898-92e7-9eaf8f0fdd75'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:33.095011 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8734f3d1-2c6b-4898-92e7-9eaf8f0fdd75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6fe8b6c27e743eaad5fe6ffc8a40e81', 'request_id': 'req-8734f3d1-2c6b-4898-92e7-9eaf8f0fdd75'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:33.108264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:33.160600 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8734f3d1-2c6b-4898-92e7-9eaf8f0fdd75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6fe8b6c27e743eaad5fe6ffc8a40e81', 'request_id': 'req-8734f3d1-2c6b-4898-92e7-9eaf8f0fdd75'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:33.182748 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8734f3d1-2c6b-4898-92e7-9eaf8f0fdd75 admin admin] The token cache is being invalidate because project d6fe8b6c27e743eaad5fe6ffc8a40e81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:16:33.183054 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8734f3d1-2c6b-4898-92e7-9eaf8f0fdd75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:33.183767 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8734f3d1-2c6b-4898-92e7-9eaf8f0fdd75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:33.186719 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1209/2398] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:16:32 2022] DELETE /identity/v3/projects/d6fe8b6c27e743eaad5fe6ffc8a40e81 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:16:33.188282 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9dc01b-f302-46d0-aa13-93eda0edcdb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:33.188282 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9dc01b-f302-46d0-aa13-93eda0edcdb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:33.188511 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9dc01b-f302-46d0-aa13-93eda0edcdb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:33.188696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9dc01b-f302-46d0-aa13-93eda0edcdb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:33.189444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9dc01b-f302-46d0-aa13-93eda0edcdb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:33.191148 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9dc01b-f302-46d0-aa13-93eda0edcdb1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:33.192205 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe660a1-eea5-469d-9c4c-427a320de37c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:33.192509 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe660a1-eea5-469d-9c4c-427a320de37c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:33.193322 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe660a1-eea5-469d-9c4c-427a320de37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:33.211353 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1190/2399] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:16:33.217090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9dc01b-f302-46d0-aa13-93eda0edcdb1 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:33.217090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5f9dc01b-f302-46d0-aa13-93eda0edcdb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:33.217368 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ffe660a1-eea5-469d-9c4c-427a320de37c None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:33.218950 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ffe660a1-eea5-469d-9c4c-427a320de37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:33.227667 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9dc01b-f302-46d0-aa13-93eda0edcdb1 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:33.245376 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc060f18-2e6a-4559-a550-d2e144a4e19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:33.245376 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc060f18-2e6a-4559-a550-d2e144a4e19e admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:33.245376 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc060f18-2e6a-4559-a550-d2e144a4e19e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:33.245376 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc060f18-2e6a-4559-a550-d2e144a4e19e admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:33.245699 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc060f18-2e6a-4559-a550-d2e144a4e19e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:33.249153 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc060f18-2e6a-4559-a550-d2e144a4e19e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:33.261746 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1210/2400] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:33.268231 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1191/2401] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:16:33.278059 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe660a1-eea5-469d-9c4c-427a320de37c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:33.318775 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe660a1-eea5-469d-9c4c-427a320de37c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:33.337879 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5694558-7dce-4f8f-8657-10caa3472d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:33.347931 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd0723-d265-498a-928d-299ef5401828 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:33.348327 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5694558-7dce-4f8f-8657-10caa3472d21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:33.348623 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5694558-7dce-4f8f-8657-10caa3472d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:33.349548 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd0723-d265-498a-928d-299ef5401828 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:33.349818 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd0723-d265-498a-928d-299ef5401828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:33.353076 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5694558-7dce-4f8f-8657-10caa3472d21 admin admin] PATH_INFO: `/v3/projects/9186616cf6be43ab8317e14dffd1588e/users/83ad969117844a50859ad29f218a3e51/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:33.406373 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5694558-7dce-4f8f-8657-10caa3472d21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9186616cf6be43ab8317e14dffd1588e, user_id=83ad969117844a50859ad29f218a3e51, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:33.409110 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5694558-7dce-4f8f-8657-10caa3472d21 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:33.410628 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b6bd0723-d265-498a-928d-299ef5401828 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:33.411556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b6bd0723-d265-498a-928d-299ef5401828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:33.442756 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1192/2402] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:33.449243 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd0723-d265-498a-928d-299ef5401828 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:33.463031 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd0723-d265-498a-928d-299ef5401828 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:33.472011 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f0272-cc8f-4fee-acf6-b9499f50efd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:33.472011 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f0272-cc8f-4fee-acf6-b9499f50efd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:33.472011 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f0272-cc8f-4fee-acf6-b9499f50efd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:33.472011 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f0272-cc8f-4fee-acf6-b9499f50efd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:33.472543 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300f0272-cc8f-4fee-acf6-b9499f50efd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:16:33.472543 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300f0272-cc8f-4fee-acf6-b9499f50efd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:33.472543 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300f0272-cc8f-4fee-acf6-b9499f50efd2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:33.481678 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1211/2403] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:16:33 2022] PUT /identity/v3/projects/9186616cf6be43ab8317e14dffd1588e/users/83ad969117844a50859ad29f218a3e51/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:33.495873 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6d032-3d4a-4b7c-b445-635d1250ef00 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:33.496070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6d032-3d4a-4b7c-b445-635d1250ef00 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:33.496287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6d032-3d4a-4b7c-b445-635d1250ef00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:33.520064 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1193/2404] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 13:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:16:33.521147 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f0272-cc8f-4fee-acf6-b9499f50efd2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:33.540947 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f0272-cc8f-4fee-acf6-b9499f50efd2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:33.550956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-85a6d032-3d4a-4b7c-b445-635d1250ef00 None None] MFA Rules not processed for user `83ad969117844a50859ad29f218a3e51`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:33.552226 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-85a6d032-3d4a-4b7c-b445-635d1250ef00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:33.582223 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:33.653603 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47f180-188c-481c-824a-140fdb4ac58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:33.654351 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47f180-188c-481c-824a-140fdb4ac58f admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:33.654996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47f180-188c-481c-824a-140fdb4ac58f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:33.655425 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47f180-188c-481c-824a-140fdb4ac58f admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:33.656065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe47f180-188c-481c-824a-140fdb4ac58f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:33.658426 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1212/2405] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:16:33.658567 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe47f180-188c-481c-824a-140fdb4ac58f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:33.664526 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a6d032-3d4a-4b7c-b445-635d1250ef00 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:33.680977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a6d032-3d4a-4b7c-b445-635d1250ef00 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:33.689747 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710cfbce-0765-4723-9563-7d17762a2b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:33.696943 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710cfbce-0765-4723-9563-7d17762a2b1f admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:33.696943 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710cfbce-0765-4723-9563-7d17762a2b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:33.696943 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710cfbce-0765-4723-9563-7d17762a2b1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:33.696943 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710cfbce-0765-4723-9563-7d17762a2b1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:33.696943 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710cfbce-0765-4723-9563-7d17762a2b1f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:33.760479 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1194/2406] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:16:33 2022] POST /identity/v3/projects => generated 373 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:16:33.765003 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47f180-188c-481c-824a-140fdb4ac58f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:33.786381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1213/2407] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:16:33 2022] POST /identity/v3/projects => generated 371 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:16:33.792434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710cfbce-0765-4723-9563-7d17762a2b1f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:33.908733 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47f180-188c-481c-824a-140fdb4ac58f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:33.914265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710cfbce-0765-4723-9563-7d17762a2b1f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:33.915591 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd5297-0269-461e-b534-205e68781946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:33.916396 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd5297-0269-461e-b534-205e68781946 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:33.916750 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd5297-0269-461e-b534-205e68781946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:33.917099 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd5297-0269-461e-b534-205e68781946 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:33.917829 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd5297-0269-461e-b534-205e68781946 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:33.920851 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd5297-0269-461e-b534-205e68781946 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:33.922215 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d399e1-fdb3-4176-8f3f-a1b690d6e16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:33.922763 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d399e1-fdb3-4176-8f3f-a1b690d6e16b admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:33.923042 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d399e1-fdb3-4176-8f3f-a1b690d6e16b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:33.923741 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d399e1-fdb3-4176-8f3f-a1b690d6e16b admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:33.924601 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d399e1-fdb3-4176-8f3f-a1b690d6e16b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:33.926678 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d399e1-fdb3-4176-8f3f-a1b690d6e16b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:33.943756 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:34.032071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1195/2408] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:16:33 2022] POST /identity/v3/users => generated 287 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:16:34.044926 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd5297-0269-461e-b534-205e68781946 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:34.067626 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1214/2409] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:16:33 2022] POST /identity/v3/users => generated 292 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:16:34.070780 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd5297-0269-461e-b534-205e68781946 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:34.077055 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d399e1-fdb3-4176-8f3f-a1b690d6e16b admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:34.094307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d399e1-fdb3-4176-8f3f-a1b690d6e16b admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:34.095239 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed193e-f313-4eee-9acd-5c0bd7c0d286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:34.095958 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed193e-f313-4eee-9acd-5c0bd7c0d286 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:34.096302 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed193e-f313-4eee-9acd-5c0bd7c0d286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:34.096769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed193e-f313-4eee-9acd-5c0bd7c0d286 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:34.097494 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ed193e-f313-4eee-9acd-5c0bd7c0d286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:34.099214 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ed193e-f313-4eee-9acd-5c0bd7c0d286 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:34.110357 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b1283-304e-44d3-a22d-ec7329848d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:34.110983 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b1283-304e-44d3-a22d-ec7329848d3c admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:34.111849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b1283-304e-44d3-a22d-ec7329848d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:34.112331 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b1283-304e-44d3-a22d-ec7329848d3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:34.113290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b1283-304e-44d3-a22d-ec7329848d3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:34.114388 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1196/2410] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:16:34.117830 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b1283-304e-44d3-a22d-ec7329848d3c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:34.118714 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed193e-f313-4eee-9acd-5c0bd7c0d286 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:34.137560 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed193e-f313-4eee-9acd-5c0bd7c0d286 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:34.143469 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1215/2411] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:16:34.144406 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b1283-304e-44d3-a22d-ec7329848d3c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:34.153389 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72264036-5b81-48c7-a3b0-11b6352e1c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:34.156157 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72264036-5b81-48c7-a3b0-11b6352e1c64 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:34.158820 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72264036-5b81-48c7-a3b0-11b6352e1c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:34.158820 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72264036-5b81-48c7-a3b0-11b6352e1c64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:34.158820 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72264036-5b81-48c7-a3b0-11b6352e1c64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:34.159035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b1283-304e-44d3-a22d-ec7329848d3c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:34.160323 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72264036-5b81-48c7-a3b0-11b6352e1c64 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:34.165437 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270221a-44b3-4e69-8c1a-1b4be7903ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:34.166178 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270221a-44b3-4e69-8c1a-1b4be7903ed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:34.166577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270221a-44b3-4e69-8c1a-1b4be7903ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:34.166924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270221a-44b3-4e69-8c1a-1b4be7903ed7 admin admin] PATH_INFO: `/v3/projects/b6773ab268fd4212afd6921805cf4181/users/50c38bf5e3f94ab2bf749b4fcaf93f48/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:34.189055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1197/2412] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:16:34.191935 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72264036-5b81-48c7-a3b0-11b6352e1c64 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:34.194332 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270221a-44b3-4e69-8c1a-1b4be7903ed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6773ab268fd4212afd6921805cf4181, user_id=50c38bf5e3f94ab2bf749b4fcaf93f48, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:34.199412 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270221a-44b3-4e69-8c1a-1b4be7903ed7 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:34.206708 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72264036-5b81-48c7-a3b0-11b6352e1c64 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:34.214740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34facb-5af5-43a6-9ec6-6d5cb184c743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:34.215582 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34facb-5af5-43a6-9ec6-6d5cb184c743 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:34.215912 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34facb-5af5-43a6-9ec6-6d5cb184c743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:34.216259 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34facb-5af5-43a6-9ec6-6d5cb184c743 admin admin] PATH_INFO: `/v3/projects/99c6da6b75574975bf36d8d3038913e8/users/9601bade2d3a48b4a731428f9d1c939e/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:34.238250 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df34facb-5af5-43a6-9ec6-6d5cb184c743 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99c6da6b75574975bf36d8d3038913e8, user_id=9601bade2d3a48b4a731428f9d1c939e, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:34.241364 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df34facb-5af5-43a6-9ec6-6d5cb184c743 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:34.321311 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1216/2413] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:16:34 2022] PUT /identity/v3/projects/b6773ab268fd4212afd6921805cf4181/users/50c38bf5e3f94ab2bf749b4fcaf93f48/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:34.321542 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1198/2414] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:16:34 2022] PUT /identity/v3/projects/99c6da6b75574975bf36d8d3038913e8/users/9601bade2d3a48b4a731428f9d1c939e/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:34.330030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34facb-5af5-43a6-9ec6-6d5cb184c743 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:34.333649 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a683e8-cf65-45df-bc28-5640b8b11c4d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:34.334154 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a683e8-cf65-45df-bc28-5640b8b11c4d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:34.334636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a683e8-cf65-45df-bc28-5640b8b11c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:34.388122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f7a683e8-cf65-45df-bc28-5640b8b11c4d None None] MFA Rules not processed for user `9601bade2d3a48b4a731428f9d1c939e`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:34.390005 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f7a683e8-cf65-45df-bc28-5640b8b11c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:34.408550 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34facb-5af5-43a6-9ec6-6d5cb184c743 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:34.414751 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6bcea5-6665-4eca-8b5a-59ec25c07b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:34.416269 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bcea5-6665-4eca-8b5a-59ec25c07b20 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:34.416269 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bcea5-6665-4eca-8b5a-59ec25c07b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:34.416269 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bcea5-6665-4eca-8b5a-59ec25c07b20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:34.417257 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6bcea5-6665-4eca-8b5a-59ec25c07b20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:34.419912 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6bcea5-6665-4eca-8b5a-59ec25c07b20 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:34.435515 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1199/2415] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:16:34.440626 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6bcea5-6665-4eca-8b5a-59ec25c07b20 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:34.445440 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:34.457240 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6bcea5-6665-4eca-8b5a-59ec25c07b20 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:34.462528 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc34d80c-105b-4d4d-8211-3917e2b974fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:34.464039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34d80c-105b-4d4d-8211-3917e2b974fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:34.464397 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34d80c-105b-4d4d-8211-3917e2b974fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:34.464749 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34d80c-105b-4d4d-8211-3917e2b974fb admin admin] PATH_INFO: `/v3/projects/b6773ab268fd4212afd6921805cf4181/users/50c38bf5e3f94ab2bf749b4fcaf93f48/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:34.466803 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc34d80c-105b-4d4d-8211-3917e2b974fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6773ab268fd4212afd6921805cf4181, user_id=50c38bf5e3f94ab2bf749b4fcaf93f48, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:34.469579 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc34d80c-105b-4d4d-8211-3917e2b974fb admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:34.481110 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1217/2416] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:34.493226 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a683e8-cf65-45df-bc28-5640b8b11c4d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:34.494061 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1200/2417] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:16:34 2022] PUT /identity/v3/projects/b6773ab268fd4212afd6921805cf4181/users/50c38bf5e3f94ab2bf749b4fcaf93f48/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:34.494733 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f7a683e8-cf65-45df-bc28-5640b8b11c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:34.507636 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b48c6f-91e9-4c9a-8903-06b79347710e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:34.508039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b48c6f-91e9-4c9a-8903-06b79347710e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:34.508314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b48c6f-91e9-4c9a-8903-06b79347710e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:34.550194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f6b48c6f-91e9-4c9a-8903-06b79347710e None None] MFA Rules not processed for user `50c38bf5e3f94ab2bf749b4fcaf93f48`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:34.552327 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f6b48c6f-91e9-4c9a-8903-06b79347710e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:34.606357 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a683e8-cf65-45df-bc28-5640b8b11c4d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:34.700395 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1201/2418] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:34.714993 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b48c6f-91e9-4c9a-8903-06b79347710e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:34.717429 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f6b48c6f-91e9-4c9a-8903-06b79347710e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:34.751640 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd690d49-bac9-4ed5-890e-c1e6fc813020 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:34.752619 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd690d49-bac9-4ed5-890e-c1e6fc813020 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:34.754753 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd690d49-bac9-4ed5-890e-c1e6fc813020 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:34.755143 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd690d49-bac9-4ed5-890e-c1e6fc813020 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:34.759313 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:34.852716 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b48c6f-91e9-4c9a-8903-06b79347710e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:34.856149 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb248d7f-b3ce-4258-90b8-b8835edcbd72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:34.856763 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb248d7f-b3ce-4258-90b8-b8835edcbd72 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:34.858371 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb248d7f-b3ce-4258-90b8-b8835edcbd72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:34.858864 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb248d7f-b3ce-4258-90b8-b8835edcbd72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:34.864798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd690d49-bac9-4ed5-890e-c1e6fc813020 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:34.868124 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd690d49-bac9-4ed5-890e-c1e6fc813020 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:34.870042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb248d7f-b3ce-4258-90b8-b8835edcbd72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:34.871726 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb248d7f-b3ce-4258-90b8-b8835edcbd72 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:34.884892 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1218/2419] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:16:34.898289 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1202/2420] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:16:35.851856 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7e192-5df8-4b07-9015-77bf01412a5e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:35.852897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7e192-5df8-4b07-9015-77bf01412a5e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:35.853417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7e192-5df8-4b07-9015-77bf01412a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:35.901044 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:35.904770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-cbb7e192-5df8-4b07-9015-77bf01412a5e None None] MFA Rules not processed for user `955679b46cd942608b7e13d09410ea01`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:35.906454 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cbb7e192-5df8-4b07-9015-77bf01412a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:36.001909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1219/2421] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:35 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:36.016725 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb248d7f-b3ce-4258-90b8-b8835edcbd72 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:36.019351 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fb248d7f-b3ce-4258-90b8-b8835edcbd72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:36.146354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb248d7f-b3ce-4258-90b8-b8835edcbd72 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:36.157884 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e414df6-e49c-41ea-b90c-c64d0b33a4f2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35dd27fd356241b885706375d880222f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:36.158517 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e414df6-e49c-41ea-b90c-c64d0b33a4f2 service glance] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:36.158795 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e414df6-e49c-41ea-b90c-c64d0b33a4f2 service glance] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:36.159070 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e414df6-e49c-41ea-b90c-c64d0b33a4f2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:36.160913 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6e414df6-e49c-41ea-b90c-c64d0b33a4f2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:36.184126 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7e192-5df8-4b07-9015-77bf01412a5e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:36.261289 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e414df6-e49c-41ea-b90c-c64d0b33a4f2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:36.266466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e414df6-e49c-41ea-b90c-c64d0b33a4f2 service glance] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:36.280724 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1203/2422] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:16:36.283792 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7e192-5df8-4b07-9015-77bf01412a5e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:36.288950 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96705b-81d9-4fe7-9111-baa28f683bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:36.289598 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96705b-81d9-4fe7-9111-baa28f683bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:36.289883 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96705b-81d9-4fe7-9111-baa28f683bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:36.290161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96705b-81d9-4fe7-9111-baa28f683bfd admin admin] PATH_INFO: `/v3/users/83ad969117844a50859ad29f218a3e51` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:36.291385 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd96705b-81d9-4fe7-9111-baa28f683bfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=83ad969117844a50859ad29f218a3e51)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:36.293402 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd96705b-81d9-4fe7-9111-baa28f683bfd admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:36.518828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-dd96705b-81d9-4fe7-9111-baa28f683bfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83ad969117844a50859ad29f218a3e51'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:36.530330 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-dd96705b-81d9-4fe7-9111-baa28f683bfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83ad969117844a50859ad29f218a3e51'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:36.577733 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96bb5e-5700-4f7f-b32e-7e5965e91757 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:36.577733 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96bb5e-5700-4f7f-b32e-7e5965e91757 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:36.577733 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96bb5e-5700-4f7f-b32e-7e5965e91757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:36.627759 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-dd96705b-81d9-4fe7-9111-baa28f683bfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83ad969117844a50859ad29f218a3e51'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:36.628566 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-dd96705b-81d9-4fe7-9111-baa28f683bfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83ad969117844a50859ad29f218a3e51'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:36.629257 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-dd96705b-81d9-4fe7-9111-baa28f683bfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83ad969117844a50859ad29f218a3e51'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:36.630982 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-bc96bb5e-5700-4f7f-b32e-7e5965e91757 None None] MFA Rules not processed for user `e42059e9af9842548ea08dcd39ff0b6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:36.632509 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bc96bb5e-5700-4f7f-b32e-7e5965e91757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:36.656177 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1220/2423] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:16:36 2022] DELETE /identity/v3/users/83ad969117844a50859ad29f218a3e51 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:16:36.656821 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:36.657469 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a37ace-bb33-4c4d-a52a-6d1643cda4d3 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:36.657751 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a37ace-bb33-4c4d-a52a-6d1643cda4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:36.658026 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a37ace-bb33-4c4d-a52a-6d1643cda4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:36.715649 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e0a37ace-bb33-4c4d-a52a-6d1643cda4d3 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:36.717554 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e0a37ace-bb33-4c4d-a52a-6d1643cda4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:36.748899 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:36.761111 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1204/2424] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 10 13:16:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:16:36.831968 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1221/2425] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:36.843942 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc96bb5e-5700-4f7f-b32e-7e5965e91757 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:36.845541 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bc96bb5e-5700-4f7f-b32e-7e5965e91757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:36.912114 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc96bb5e-5700-4f7f-b32e-7e5965e91757 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:36.975468 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc52b008-9b28-4e13-8dcd-0156f4104594 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:36.975468 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52b008-9b28-4e13-8dcd-0156f4104594 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:36.975468 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52b008-9b28-4e13-8dcd-0156f4104594 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:36.975920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52b008-9b28-4e13-8dcd-0156f4104594 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:36.975997 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a37ace-bb33-4c4d-a52a-6d1643cda4d3 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:36.977594 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e0a37ace-bb33-4c4d-a52a-6d1643cda4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:36.987965 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc52b008-9b28-4e13-8dcd-0156f4104594 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:36.989876 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc52b008-9b28-4e13-8dcd-0156f4104594 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:37.008538 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1205/2426] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:16:37.118059 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a37ace-bb33-4c4d-a52a-6d1643cda4d3 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:37.134527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88eee9-43d4-459c-adb0-04d6faaa30a5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59395bfa505c4540b3957deee78eb698', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:37.135168 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88eee9-43d4-459c-adb0-04d6faaa30a5 service swift] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:37.135694 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88eee9-43d4-459c-adb0-04d6faaa30a5 service swift] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:37.135982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88eee9-43d4-459c-adb0-04d6faaa30a5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:37.199967 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce88eee9-43d4-459c-adb0-04d6faaa30a5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:37.202326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce88eee9-43d4-459c-adb0-04d6faaa30a5 service swift] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:37.224501 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1222/2427] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 13:16:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 13:16:37.652394 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc52b008-9b28-4e13-8dcd-0156f4104594 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:37.654048 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fc52b008-9b28-4e13-8dcd-0156f4104594 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:37.675243 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc52b008-9b28-4e13-8dcd-0156f4104594 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:37.697713 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9972d84-e42a-4a60-a81d-c31df968ad27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:37.697713 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9972d84-e42a-4a60-a81d-c31df968ad27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:37.697713 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9972d84-e42a-4a60-a81d-c31df968ad27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:37.697713 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9972d84-e42a-4a60-a81d-c31df968ad27 admin admin] PATH_INFO: `/v3/projects/9186616cf6be43ab8317e14dffd1588e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:37.698110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9972d84-e42a-4a60-a81d-c31df968ad27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9186616cf6be43ab8317e14dffd1588e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:37.698110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9972d84-e42a-4a60-a81d-c31df968ad27 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:37.884186 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e9972d84-e42a-4a60-a81d-c31df968ad27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9186616cf6be43ab8317e14dffd1588e', 'request_id': 'req-e9972d84-e42a-4a60-a81d-c31df968ad27'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:37.916105 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e9972d84-e42a-4a60-a81d-c31df968ad27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9186616cf6be43ab8317e14dffd1588e', 'request_id': 'req-e9972d84-e42a-4a60-a81d-c31df968ad27'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:38.023346 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e9972d84-e42a-4a60-a81d-c31df968ad27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9186616cf6be43ab8317e14dffd1588e', 'request_id': 'req-e9972d84-e42a-4a60-a81d-c31df968ad27'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:38.026250 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:38.040623 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e9972d84-e42a-4a60-a81d-c31df968ad27 admin admin] The token cache is being invalidate because project 9186616cf6be43ab8317e14dffd1588e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:16:38.040941 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e9972d84-e42a-4a60-a81d-c31df968ad27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:38.041837 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e9972d84-e42a-4a60-a81d-c31df968ad27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:38.043804 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1206/2428] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:16:37 2022] DELETE /identity/v3/projects/9186616cf6be43ab8317e14dffd1588e => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:16:38.047992 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88eee9-43d4-459c-adb0-04d6faaa30a5 service swift] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:38.064072 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88eee9-43d4-459c-adb0-04d6faaa30a5 service swift] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:38.142172 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:38.176786 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ec175-43a1-41c9-b3ce-42285d2c0349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:38.177869 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ec175-43a1-41c9-b3ce-42285d2c0349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:38.178144 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ec175-43a1-41c9-b3ce-42285d2c0349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:38.178609 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ec175-43a1-41c9-b3ce-42285d2c0349 admin admin] PATH_INFO: `/v3/users/50c38bf5e3f94ab2bf749b4fcaf93f48` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:38.180004 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ec175-43a1-41c9-b3ce-42285d2c0349 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50c38bf5e3f94ab2bf749b4fcaf93f48)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:38.182484 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ec175-43a1-41c9-b3ce-42285d2c0349 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:38.700772 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a75ec175-43a1-41c9-b3ce-42285d2c0349 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50c38bf5e3f94ab2bf749b4fcaf93f48'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:38.718418 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a75ec175-43a1-41c9-b3ce-42285d2c0349 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50c38bf5e3f94ab2bf749b4fcaf93f48'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:38.800691 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a75ec175-43a1-41c9-b3ce-42285d2c0349 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50c38bf5e3f94ab2bf749b4fcaf93f48'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:38.801262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a75ec175-43a1-41c9-b3ce-42285d2c0349 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50c38bf5e3f94ab2bf749b4fcaf93f48'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:38.801883 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a75ec175-43a1-41c9-b3ce-42285d2c0349 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50c38bf5e3f94ab2bf749b4fcaf93f48'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:38.834993 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1223/2429] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:16:38 2022] DELETE /identity/v3/users/50c38bf5e3f94ab2bf749b4fcaf93f48 => generated 0 bytes in 787 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:38.984886 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9972d84-e42a-4a60-a81d-c31df968ad27 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:38.987105 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e9972d84-e42a-4a60-a81d-c31df968ad27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:39.004620 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9972d84-e42a-4a60-a81d-c31df968ad27 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:39.079863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880dbb0-a3f0-4d86-8303-4811e3037ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:39.080521 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880dbb0-a3f0-4d86-8303-4811e3037ee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:39.080804 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880dbb0-a3f0-4d86-8303-4811e3037ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:39.081069 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880dbb0-a3f0-4d86-8303-4811e3037ee9 admin admin] PATH_INFO: `/v3/users/9601bade2d3a48b4a731428f9d1c939e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:39.092732 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c880dbb0-a3f0-4d86-8303-4811e3037ee9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9601bade2d3a48b4a731428f9d1c939e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:39.096987 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c880dbb0-a3f0-4d86-8303-4811e3037ee9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:39.291095 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c880dbb0-a3f0-4d86-8303-4811e3037ee9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9601bade2d3a48b4a731428f9d1c939e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:39.417081 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c880dbb0-a3f0-4d86-8303-4811e3037ee9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9601bade2d3a48b4a731428f9d1c939e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:39.418191 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c880dbb0-a3f0-4d86-8303-4811e3037ee9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9601bade2d3a48b4a731428f9d1c939e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:39.419147 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c880dbb0-a3f0-4d86-8303-4811e3037ee9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9601bade2d3a48b4a731428f9d1c939e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:39.431754 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c880dbb0-a3f0-4d86-8303-4811e3037ee9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9601bade2d3a48b4a731428f9d1c939e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:39.482128 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1207/2430] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:16:38 2022] DELETE /identity/v3/users/9601bade2d3a48b4a731428f9d1c939e => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:39.482302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d7cff-599a-47bc-939b-5d4effe1e67a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:39.483623 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d7cff-599a-47bc-939b-5d4effe1e67a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:39.483924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d7cff-599a-47bc-939b-5d4effe1e67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:39.510502 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880dbb0-a3f0-4d86-8303-4811e3037ee9 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:39.511920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c880dbb0-a3f0-4d86-8303-4811e3037ee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:39.517567 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5d6d7cff-599a-47bc-939b-5d4effe1e67a None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:39.518794 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5d6d7cff-599a-47bc-939b-5d4effe1e67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:39.645201 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1224/2431] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:39.661761 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d7cff-599a-47bc-939b-5d4effe1e67a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:39.661761 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5d6d7cff-599a-47bc-939b-5d4effe1e67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:39.663278 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880dbb0-a3f0-4d86-8303-4811e3037ee9 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:39.676581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80048d-9a4d-4752-9200-47fb465489ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:39.678945 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80048d-9a4d-4752-9200-47fb465489ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:39.679259 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80048d-9a4d-4752-9200-47fb465489ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:39.679545 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80048d-9a4d-4752-9200-47fb465489ef admin admin] PATH_INFO: `/v3/projects/b6773ab268fd4212afd6921805cf4181` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:39.684179 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c80048d-9a4d-4752-9200-47fb465489ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6773ab268fd4212afd6921805cf4181)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:39.689541 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c80048d-9a4d-4752-9200-47fb465489ef admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:39.740929 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:39.752422 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d7cff-599a-47bc-939b-5d4effe1e67a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:39.762090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f823cc-924d-488e-ae04-9fa7309a01dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:39.762908 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f823cc-924d-488e-ae04-9fa7309a01dc service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:39.763355 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f823cc-924d-488e-ae04-9fa7309a01dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:39.763731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f823cc-924d-488e-ae04-9fa7309a01dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:39.779451 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f823cc-924d-488e-ae04-9fa7309a01dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:39.783289 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f823cc-924d-488e-ae04-9fa7309a01dc service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:39.799252 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1225/2432] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:16:39.858294 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0c80048d-9a4d-4752-9200-47fb465489ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6773ab268fd4212afd6921805cf4181', 'request_id': 'req-0c80048d-9a4d-4752-9200-47fb465489ef'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:39.872100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0c80048d-9a4d-4752-9200-47fb465489ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6773ab268fd4212afd6921805cf4181', 'request_id': 'req-0c80048d-9a4d-4752-9200-47fb465489ef'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:39.987385 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0c80048d-9a4d-4752-9200-47fb465489ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6773ab268fd4212afd6921805cf4181', 'request_id': 'req-0c80048d-9a4d-4752-9200-47fb465489ef'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:40.003284 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0c80048d-9a4d-4752-9200-47fb465489ef admin admin] The token cache is being invalidate because project b6773ab268fd4212afd6921805cf4181 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:16:40.007732 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0c80048d-9a4d-4752-9200-47fb465489ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:40.011962 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0c80048d-9a4d-4752-9200-47fb465489ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:40.015617 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1208/2433] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:16:39 2022] DELETE /identity/v3/projects/b6773ab268fd4212afd6921805cf4181 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:16:40.024188 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b8ca7-e0a5-4cde-922c-8db76f4d04d9 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:40.024472 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b8ca7-e0a5-4cde-922c-8db76f4d04d9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:40.024748 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b8ca7-e0a5-4cde-922c-8db76f4d04d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:40.035375 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:40.084874 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a40b8ca7-e0a5-4cde-922c-8db76f4d04d9 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:40.086127 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a40b8ca7-e0a5-4cde-922c-8db76f4d04d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:40.264370 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1226/2434] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:40.272438 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26900ff0-243e-44fc-ac2b-1e56a1b77444 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:40.272724 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26900ff0-243e-44fc-ac2b-1e56a1b77444 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:40.272998 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26900ff0-243e-44fc-ac2b-1e56a1b77444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:40.331507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-26900ff0-243e-44fc-ac2b-1e56a1b77444 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:40.331507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-26900ff0-243e-44fc-ac2b-1e56a1b77444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:40.343994 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b8ca7-e0a5-4cde-922c-8db76f4d04d9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:40.351086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1209/2435] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:40.354994 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b8ca7-e0a5-4cde-922c-8db76f4d04d9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:40.360409 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e7346-2533-489c-b205-8bc7218bdfee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:40.361093 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e7346-2533-489c-b205-8bc7218bdfee admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:40.361723 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e7346-2533-489c-b205-8bc7218bdfee admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:40.362067 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e7346-2533-489c-b205-8bc7218bdfee admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:40.362789 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e7346-2533-489c-b205-8bc7218bdfee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:16:40.363158 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e7346-2533-489c-b205-8bc7218bdfee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:40.364885 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e7346-2533-489c-b205-8bc7218bdfee admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:40.385570 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26900ff0-243e-44fc-ac2b-1e56a1b77444 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:40.392479 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1227/2436] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 13:16:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:16:40.397679 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26900ff0-243e-44fc-ac2b-1e56a1b77444 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:40.402925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5f52e-04dd-4fc7-a11b-7060cf62fa15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:40.403673 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5f52e-04dd-4fc7-a11b-7060cf62fa15 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:40.404025 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5f52e-04dd-4fc7-a11b-7060cf62fa15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:40.404367 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5f52e-04dd-4fc7-a11b-7060cf62fa15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:40.405144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5f52e-04dd-4fc7-a11b-7060cf62fa15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:40.407136 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5f52e-04dd-4fc7-a11b-7060cf62fa15 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:40.442440 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e7346-2533-489c-b205-8bc7218bdfee admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:40.444402 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e88e7346-2533-489c-b205-8bc7218bdfee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:40.463834 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e7346-2533-489c-b205-8bc7218bdfee admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:40.468291 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e8d5a1-ffc9-4953-936e-b056d2c19a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:40.469137 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8d5a1-ffc9-4953-936e-b056d2c19a85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:40.469646 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8d5a1-ffc9-4953-936e-b056d2c19a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:40.470225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8d5a1-ffc9-4953-936e-b056d2c19a85 admin admin] PATH_INFO: `/v3/projects/99c6da6b75574975bf36d8d3038913e8` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:40.476011 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e8d5a1-ffc9-4953-936e-b056d2c19a85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99c6da6b75574975bf36d8d3038913e8)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:40.478076 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e8d5a1-ffc9-4953-936e-b056d2c19a85 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:40.485402 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1210/2437] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:16:40 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:16:40.490053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5f52e-04dd-4fc7-a11b-7060cf62fa15 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:40.594863 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-73e8d5a1-ffc9-4953-936e-b056d2c19a85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99c6da6b75574975bf36d8d3038913e8', 'request_id': 'req-73e8d5a1-ffc9-4953-936e-b056d2c19a85'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:40.605954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5f52e-04dd-4fc7-a11b-7060cf62fa15 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:40.613588 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-73e8d5a1-ffc9-4953-936e-b056d2c19a85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99c6da6b75574975bf36d8d3038913e8', 'request_id': 'req-73e8d5a1-ffc9-4953-936e-b056d2c19a85'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:40.696569 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-73e8d5a1-ffc9-4953-936e-b056d2c19a85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99c6da6b75574975bf36d8d3038913e8', 'request_id': 'req-73e8d5a1-ffc9-4953-936e-b056d2c19a85'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:40.720726 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-73e8d5a1-ffc9-4953-936e-b056d2c19a85 admin admin] The token cache is being invalidate because project 99c6da6b75574975bf36d8d3038913e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:16:40.720726 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-73e8d5a1-ffc9-4953-936e-b056d2c19a85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:40.722596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-73e8d5a1-ffc9-4953-936e-b056d2c19a85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:40.725316 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1228/2438] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:16:40 2022] DELETE /identity/v3/projects/99c6da6b75574975bf36d8d3038913e8 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:40.729171 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85638131-8d1e-41d8-9100-fd11b7b8ad18 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:40.729577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85638131-8d1e-41d8-9100-fd11b7b8ad18 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:40.729920 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85638131-8d1e-41d8-9100-fd11b7b8ad18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:40.731713 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb5afe-0c9c-4d30-9fe4-07c35557fad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:40.733503 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb5afe-0c9c-4d30-9fe4-07c35557fad5 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:40.734007 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb5afe-0c9c-4d30-9fe4-07c35557fad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:40.734352 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb5afe-0c9c-4d30-9fe4-07c35557fad5 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:40.740138 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb5afe-0c9c-4d30-9fe4-07c35557fad5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:40.746318 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb5afe-0c9c-4d30-9fe4-07c35557fad5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:40.759413 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-85638131-8d1e-41d8-9100-fd11b7b8ad18 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:40.761052 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-85638131-8d1e-41d8-9100-fd11b7b8ad18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:40.787825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1229/2439] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:40.793411 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678b764-4d67-45ac-b47d-be9d6cf8ab4c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:40.793701 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678b764-4d67-45ac-b47d-be9d6cf8ab4c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:40.793966 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678b764-4d67-45ac-b47d-be9d6cf8ab4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:40.832544 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-4678b764-4d67-45ac-b47d-be9d6cf8ab4c None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:40.833839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4678b764-4d67-45ac-b47d-be9d6cf8ab4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:40.853350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1211/2440] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:16:40 2022] POST /identity/v3/users => generated 288 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:16:40.855581 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1230/2441] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:40.861855 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb5afe-0c9c-4d30-9fe4-07c35557fad5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:40.865039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f8eb5afe-0c9c-4d30-9fe4-07c35557fad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:40.867806 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678b764-4d67-45ac-b47d-be9d6cf8ab4c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:40.890024 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678b764-4d67-45ac-b47d-be9d6cf8ab4c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:40.893428 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb5afe-0c9c-4d30-9fe4-07c35557fad5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:40.900958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d2c1f0-6a57-4667-9475-733746adbb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:40.901579 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2c1f0-6a57-4667-9475-733746adbb91 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:40.901856 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2c1f0-6a57-4667-9475-733746adbb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:40.902125 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2c1f0-6a57-4667-9475-733746adbb91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:40.902777 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d2c1f0-6a57-4667-9475-733746adbb91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:16:40.903079 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d2c1f0-6a57-4667-9475-733746adbb91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:40.905681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d61caa9-a11b-4079-9c32-03158be60417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:40.906419 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61caa9-a11b-4079-9c32-03158be60417 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:40.908446 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61caa9-a11b-4079-9c32-03158be60417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:40.908724 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61caa9-a11b-4079-9c32-03158be60417 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:40.909387 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d61caa9-a11b-4079-9c32-03158be60417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:40.911002 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d61caa9-a11b-4079-9c32-03158be60417 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:40.913722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d2c1f0-6a57-4667-9475-733746adbb91 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:40.933755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1212/2442] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:16:40.938658 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1231/2443] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 13:16:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:16:40.943496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d61caa9-a11b-4079-9c32-03158be60417 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:40.945939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d2c1f0-6a57-4667-9475-733746adbb91 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:40.959068 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:40.970882 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d2c1f0-6a57-4667-9475-733746adbb91 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:40.971734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d61caa9-a11b-4079-9c32-03158be60417 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:40.979093 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3687765-8e83-4245-ab43-a3d53ec8fc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:40.980601 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3687765-8e83-4245-ab43-a3d53ec8fc4a admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:40.981448 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea387af3-4c67-4158-a78a-9f4e2950bd35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:40.982150 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea387af3-4c67-4158-a78a-9f4e2950bd35 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:40.982682 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3687765-8e83-4245-ab43-a3d53ec8fc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:40.983130 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea387af3-4c67-4158-a78a-9f4e2950bd35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:40.984105 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3687765-8e83-4245-ab43-a3d53ec8fc4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:40.984507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea387af3-4c67-4158-a78a-9f4e2950bd35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:40.985275 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea387af3-4c67-4158-a78a-9f4e2950bd35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:40.986044 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3687765-8e83-4245-ab43-a3d53ec8fc4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:40.991977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3687765-8e83-4245-ab43-a3d53ec8fc4a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:40.992209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea387af3-4c67-4158-a78a-9f4e2950bd35 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:41.016442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1213/2444] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:16:41.020372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea387af3-4c67-4158-a78a-9f4e2950bd35 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:41.036667 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea387af3-4c67-4158-a78a-9f4e2950bd35 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:41.042904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f58de-eb86-40c9-aba1-2d4631566e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:41.043829 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f58de-eb86-40c9-aba1-2d4631566e84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:41.044215 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f58de-eb86-40c9-aba1-2d4631566e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:41.044631 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f58de-eb86-40c9-aba1-2d4631566e84 admin admin] PATH_INFO: `/v3/projects/cd8a24b99e604a68bb968f47897306d1/users/afd1122093f14d1f92ddaccfac47eb39/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:41.078723 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4f58de-eb86-40c9-aba1-2d4631566e84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd8a24b99e604a68bb968f47897306d1, user_id=afd1122093f14d1f92ddaccfac47eb39, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:41.083297 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4f58de-eb86-40c9-aba1-2d4631566e84 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:41.151218 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1232/2445] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:16:40 2022] POST /identity/v3/projects => generated 405 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:16:41.153965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3687765-8e83-4245-ab43-a3d53ec8fc4a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:41.183060 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1214/2446] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:16:41 2022] PUT /identity/v3/projects/cd8a24b99e604a68bb968f47897306d1/users/afd1122093f14d1f92ddaccfac47eb39/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:41.185322 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366b935-0841-42af-8d6b-0f0f1ec0c8bc None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:41.185624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366b935-0841-42af-8d6b-0f0f1ec0c8bc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:41.186132 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366b935-0841-42af-8d6b-0f0f1ec0c8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:41.240450 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:41.266321 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e366b935-0841-42af-8d6b-0f0f1ec0c8bc None None] MFA Rules not processed for user `afd1122093f14d1f92ddaccfac47eb39`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:41.271018 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e366b935-0841-42af-8d6b-0f0f1ec0c8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:41.328956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3687765-8e83-4245-ab43-a3d53ec8fc4a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:41.415852 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1215/2447] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:41.416891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366b935-0841-42af-8d6b-0f0f1ec0c8bc None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:41.442513 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ec582b-9f7c-4d3f-8f03-2f8f975b4ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:41.443139 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec582b-9f7c-4d3f-8f03-2f8f975b4ed3 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:41.443455 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec582b-9f7c-4d3f-8f03-2f8f975b4ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:41.443729 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec582b-9f7c-4d3f-8f03-2f8f975b4ed3 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:41.444442 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ec582b-9f7c-4d3f-8f03-2f8f975b4ed3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:41.446396 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ec582b-9f7c-4d3f-8f03-2f8f975b4ed3 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:41.535950 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366b935-0841-42af-8d6b-0f0f1ec0c8bc None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:41.547129 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0fb90d-8712-4630-b7e5-ae6198574ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:41.547872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0fb90d-8712-4630-b7e5-ae6198574ee2 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:41.548157 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0fb90d-8712-4630-b7e5-ae6198574ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:41.548432 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0fb90d-8712-4630-b7e5-ae6198574ee2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:41.549072 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0fb90d-8712-4630-b7e5-ae6198574ee2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:41.551442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0fb90d-8712-4630-b7e5-ae6198574ee2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:41.556844 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1233/2448] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:16:41 2022] POST /identity/v3/users => generated 303 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:16:41.558181 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ec582b-9f7c-4d3f-8f03-2f8f975b4ed3 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:41.570588 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ec582b-9f7c-4d3f-8f03-2f8f975b4ed3 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:41.581751 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6bb8e5-178c-46de-9799-c57d983651a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:41.582365 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6bb8e5-178c-46de-9799-c57d983651a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:41.582641 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6bb8e5-178c-46de-9799-c57d983651a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:41.582920 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6bb8e5-178c-46de-9799-c57d983651a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:41.583597 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6bb8e5-178c-46de-9799-c57d983651a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:41.585179 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6bb8e5-178c-46de-9799-c57d983651a7 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:41.605959 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1234/2449] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:16:41.610013 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6bb8e5-178c-46de-9799-c57d983651a7 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:41.620692 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6bb8e5-178c-46de-9799-c57d983651a7 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:41.632120 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a2ecb-1633-4d3c-a65a-9202bdb29f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:41.633542 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a2ecb-1633-4d3c-a65a-9202bdb29f1d admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:41.633892 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a2ecb-1633-4d3c-a65a-9202bdb29f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:41.634718 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a2ecb-1633-4d3c-a65a-9202bdb29f1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:41.635481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61a2ecb-1633-4d3c-a65a-9202bdb29f1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:41.638861 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61a2ecb-1633-4d3c-a65a-9202bdb29f1d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:41.649452 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1216/2450] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:16:41 2022] POST /identity/v3/projects => generated 375 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:16:41.654248 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1235/2451] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:16:41.654994 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0fb90d-8712-4630-b7e5-ae6198574ee2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:41.658044 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a2ecb-1633-4d3c-a65a-9202bdb29f1d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:41.668633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a2ecb-1633-4d3c-a65a-9202bdb29f1d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:41.736128 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:41.757067 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f687ed-5d46-46d3-bacb-9f9de23bf63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:41.758135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f687ed-5d46-46d3-bacb-9f9de23bf63a admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:41.758570 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f687ed-5d46-46d3-bacb-9f9de23bf63a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:41.759097 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f687ed-5d46-46d3-bacb-9f9de23bf63a admin admin] PATH_INFO: `/v3/projects/477a1089a7864481b1f9d2eded4bc057/users/ef102be9234f4003adb7856b2a1308da/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:41.779144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0fb90d-8712-4630-b7e5-ae6198574ee2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:41.782912 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703c2bd-16af-4bec-b2f4-c55b633f49ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:41.783568 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703c2bd-16af-4bec-b2f4-c55b633f49ea admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:41.783847 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703c2bd-16af-4bec-b2f4-c55b633f49ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:41.784118 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703c2bd-16af-4bec-b2f4-c55b633f49ea admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:41.785155 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2703c2bd-16af-4bec-b2f4-c55b633f49ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:41.786260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f687ed-5d46-46d3-bacb-9f9de23bf63a admin admin] RBAC: Authorizing `identity:create_grant(project_id=477a1089a7864481b1f9d2eded4bc057, user_id=ef102be9234f4003adb7856b2a1308da, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:41.787611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2703c2bd-16af-4bec-b2f4-c55b633f49ea admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:41.789366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f687ed-5d46-46d3-bacb-9f9de23bf63a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:41.869262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1236/2452] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:16:41 2022] PUT /identity/v3/projects/477a1089a7864481b1f9d2eded4bc057/users/ef102be9234f4003adb7856b2a1308da/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:41.880375 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78a4d6-3865-4935-bf61-317cfd88709e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:41.880724 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78a4d6-3865-4935-bf61-317cfd88709e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:41.880923 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78a4d6-3865-4935-bf61-317cfd88709e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:41.892087 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1217/2453] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:16:41 2022] POST /identity/v3/users => generated 294 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:16:41.896363 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703c2bd-16af-4bec-b2f4-c55b633f49ea admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:41.911939 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703c2bd-16af-4bec-b2f4-c55b633f49ea admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:41.947251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6b78a4d6-3865-4935-bf61-317cfd88709e None None] MFA Rules not processed for user `ef102be9234f4003adb7856b2a1308da`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:41.947251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6b78a4d6-3865-4935-bf61-317cfd88709e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:41.970107 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:42.022161 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19ca2f-6bff-452f-ac77-70f136f9a7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:42.022901 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19ca2f-6bff-452f-ac77-70f136f9a7af admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:42.023088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19ca2f-6bff-452f-ac77-70f136f9a7af admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:42.023396 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19ca2f-6bff-452f-ac77-70f136f9a7af admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:42.024064 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d19ca2f-6bff-452f-ac77-70f136f9a7af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:42.025730 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d19ca2f-6bff-452f-ac77-70f136f9a7af admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:42.051590 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1218/2454] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:16:42.057313 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19ca2f-6bff-452f-ac77-70f136f9a7af admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:42.070043 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19ca2f-6bff-452f-ac77-70f136f9a7af admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:42.074772 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1237/2455] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:41 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:16:42.077860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7b876-974b-4bb9-9a05-330eb4c9c2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:42.078656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7b876-974b-4bb9-9a05-330eb4c9c2fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:42.079087 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7b876-974b-4bb9-9a05-330eb4c9c2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:42.079488 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7b876-974b-4bb9-9a05-330eb4c9c2fd admin admin] PATH_INFO: `/v3/projects/59a729419e70483fbde2ae60ebda5df8/users/8a7641e74e3946c599a8bf8aa3125f81/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:42.087216 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b78a4d6-3865-4935-bf61-317cfd88709e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:42.102147 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd7b876-974b-4bb9-9a05-330eb4c9c2fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=59a729419e70483fbde2ae60ebda5df8, user_id=8a7641e74e3946c599a8bf8aa3125f81, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:42.105291 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd7b876-974b-4bb9-9a05-330eb4c9c2fd admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:42.133896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b78a4d6-3865-4935-bf61-317cfd88709e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:42.133896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4b061-9191-4fd5-99ce-df4f500fd46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:42.133896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4b061-9191-4fd5-99ce-df4f500fd46f admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:42.133896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4b061-9191-4fd5-99ce-df4f500fd46f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:42.134305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4b061-9191-4fd5-99ce-df4f500fd46f admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:42.134305 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b4b061-9191-4fd5-99ce-df4f500fd46f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:42.134639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b4b061-9191-4fd5-99ce-df4f500fd46f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:42.384352 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1219/2457] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:16:42 2022] PUT /identity/v3/projects/59a729419e70483fbde2ae60ebda5df8/users/8a7641e74e3946c599a8bf8aa3125f81/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:42.387848 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1238/2457] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:16:42 2022] POST /identity/v3/projects => generated 405 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:16:42.389886 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4b061-9191-4fd5-99ce-df4f500fd46f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:42.390215 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7b876-974b-4bb9-9a05-330eb4c9c2fd admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:42.510626 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4b061-9191-4fd5-99ce-df4f500fd46f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:42.517280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0c89d-0597-479a-b74e-136465fab3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:42.517896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0c89d-0597-479a-b74e-136465fab3de admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:42.518287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0c89d-0597-479a-b74e-136465fab3de admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:42.518559 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0c89d-0597-479a-b74e-136465fab3de admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:42.519311 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f0c89d-0597-479a-b74e-136465fab3de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:42.523768 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f0c89d-0597-479a-b74e-136465fab3de admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:42.523883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7b876-974b-4bb9-9a05-330eb4c9c2fd admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:42.529126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ee7e1-d85e-4d77-b619-7caf575c304f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:42.529840 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ee7e1-d85e-4d77-b619-7caf575c304f admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:42.530164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ee7e1-d85e-4d77-b619-7caf575c304f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:42.530543 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ee7e1-d85e-4d77-b619-7caf575c304f admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:42.531208 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ee7e1-d85e-4d77-b619-7caf575c304f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:42.533403 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ee7e1-d85e-4d77-b619-7caf575c304f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:42.534999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1239/2458] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:16:42.541420 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0c89d-0597-479a-b74e-136465fab3de admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:42.565588 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0c89d-0597-479a-b74e-136465fab3de admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:42.574385 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b1c3b-5c56-4e1e-a11d-267f13206760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:42.575361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b1c3b-5c56-4e1e-a11d-267f13206760 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:42.575717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b1c3b-5c56-4e1e-a11d-267f13206760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:42.576068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b1c3b-5c56-4e1e-a11d-267f13206760 admin admin] PATH_INFO: `/v3/projects/59a729419e70483fbde2ae60ebda5df8/users/8a7641e74e3946c599a8bf8aa3125f81/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:42.578745 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b1c3b-5c56-4e1e-a11d-267f13206760 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59a729419e70483fbde2ae60ebda5df8, user_id=8a7641e74e3946c599a8bf8aa3125f81, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:42.581703 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b1c3b-5c56-4e1e-a11d-267f13206760 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:42.593898 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1240/2459] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:16:42 2022] PUT /identity/v3/projects/59a729419e70483fbde2ae60ebda5df8/users/8a7641e74e3946c599a8bf8aa3125f81/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:42.605300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2142d82-4c7b-4052-b874-07f9b32538ad None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:42.605593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2142d82-4c7b-4052-b874-07f9b32538ad None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:42.605873 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2142d82-4c7b-4052-b874-07f9b32538ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:42.630287 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1220/2460] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:16:42 2022] POST /identity/v3/users => generated 309 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:16:42.634693 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ee7e1-d85e-4d77-b619-7caf575c304f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:42.646205 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ee7e1-d85e-4d77-b619-7caf575c304f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:42.672222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e2142d82-4c7b-4052-b874-07f9b32538ad None None] MFA Rules not processed for user `8a7641e74e3946c599a8bf8aa3125f81`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:42.673262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e2142d82-4c7b-4052-b874-07f9b32538ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:42.713612 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:42.783349 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb70f1-18af-499c-b502-91f56a3d6199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:42.784126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb70f1-18af-499c-b502-91f56a3d6199 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:42.784528 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb70f1-18af-499c-b502-91f56a3d6199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:42.784812 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb70f1-18af-499c-b502-91f56a3d6199 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:42.785602 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb70f1-18af-499c-b502-91f56a3d6199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:42.787705 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb70f1-18af-499c-b502-91f56a3d6199 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:42.803230 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1221/2461] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:16:42.807667 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb70f1-18af-499c-b502-91f56a3d6199 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:42.809352 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:42.819436 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1241/2462] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:16:42.827954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb70f1-18af-499c-b502-91f56a3d6199 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:42.836542 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2142d82-4c7b-4052-b874-07f9b32538ad None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:42.836542 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e2142d82-4c7b-4052-b874-07f9b32538ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:42.836740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27b84c-0ca8-4849-9217-f9700778b4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:42.836740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27b84c-0ca8-4849-9217-f9700778b4f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:42.836740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27b84c-0ca8-4849-9217-f9700778b4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:42.836740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27b84c-0ca8-4849-9217-f9700778b4f1 admin admin] PATH_INFO: `/v3/projects/dd400459c29647b3b5b9522bb2374354/users/dbdb2f11a4e5448bb3f6ee81df3771e0/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:42.853909 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb27b84c-0ca8-4849-9217-f9700778b4f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd400459c29647b3b5b9522bb2374354, user_id=dbdb2f11a4e5448bb3f6ee81df3771e0, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:42.856982 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb27b84c-0ca8-4849-9217-f9700778b4f1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:42.902139 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2142d82-4c7b-4052-b874-07f9b32538ad None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:42.907244 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01bf3d-8489-4c71-8e57-3d4a8a724612 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:42.908181 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01bf3d-8489-4c71-8e57-3d4a8a724612 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:42.908605 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01bf3d-8489-4c71-8e57-3d4a8a724612 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:42.909023 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01bf3d-8489-4c71-8e57-3d4a8a724612 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:42.932686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1222/2463] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:16:42 2022] PUT /identity/v3/projects/dd400459c29647b3b5b9522bb2374354/users/dbdb2f11a4e5448bb3f6ee81df3771e0/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:42.936405 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27b84c-0ca8-4849-9217-f9700778b4f1 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:42.985672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01bf3d-8489-4c71-8e57-3d4a8a724612 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:42.989300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01bf3d-8489-4c71-8e57-3d4a8a724612 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:43.006286 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1242/2464] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:16:43.054784 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27b84c-0ca8-4849-9217-f9700778b4f1 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:43.063370 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39335235-042d-49ca-b278-5316f8c7bfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:43.064055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39335235-042d-49ca-b278-5316f8c7bfdc admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:43.064607 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39335235-042d-49ca-b278-5316f8c7bfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:43.064890 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39335235-042d-49ca-b278-5316f8c7bfdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:43.065857 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39335235-042d-49ca-b278-5316f8c7bfdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:43.068961 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39335235-042d-49ca-b278-5316f8c7bfdc admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:43.081463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1223/2465] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:16:43.086605 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01bf3d-8489-4c71-8e57-3d4a8a724612 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:43.098230 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01bf3d-8489-4c71-8e57-3d4a8a724612 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:43.108738 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf0c1c-470c-4d87-a163-be7135962580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:43.109409 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf0c1c-470c-4d87-a163-be7135962580 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:43.109690 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf0c1c-470c-4d87-a163-be7135962580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:43.109960 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf0c1c-470c-4d87-a163-be7135962580 admin admin] PATH_INFO: `/v3/projects/dd400459c29647b3b5b9522bb2374354/users/dbdb2f11a4e5448bb3f6ee81df3771e0/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:43.112382 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf0c1c-470c-4d87-a163-be7135962580 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd400459c29647b3b5b9522bb2374354, user_id=dbdb2f11a4e5448bb3f6ee81df3771e0, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:43.114998 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf0c1c-470c-4d87-a163-be7135962580 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:43.137489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1243/2466] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:16:43 2022] PUT /identity/v3/projects/dd400459c29647b3b5b9522bb2374354/users/dbdb2f11a4e5448bb3f6ee81df3771e0/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:43.139653 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698b0be-04bd-423b-a557-a7f0d2ff3f17 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:43.140143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698b0be-04bd-423b-a557-a7f0d2ff3f17 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:43.140497 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698b0be-04bd-423b-a557-a7f0d2ff3f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:43.180232 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c698b0be-04bd-423b-a557-a7f0d2ff3f17 None None] MFA Rules not processed for user `dbdb2f11a4e5448bb3f6ee81df3771e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:43.181825 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c698b0be-04bd-423b-a557-a7f0d2ff3f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:43.272832 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1224/2467] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:43 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:43.278876 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf0c1c-470c-4d87-a163-be7135962580 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:43.281683 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-38bf0c1c-470c-4d87-a163-be7135962580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:43.361204 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf0c1c-470c-4d87-a163-be7135962580 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:43.366187 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018cc0ea-912d-4dee-b4bd-752c744476cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:43.366895 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018cc0ea-912d-4dee-b4bd-752c744476cb service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:43.367272 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018cc0ea-912d-4dee-b4bd-752c744476cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:43.367707 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018cc0ea-912d-4dee-b4bd-752c744476cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:43.380366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018cc0ea-912d-4dee-b4bd-752c744476cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:43.381862 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018cc0ea-912d-4dee-b4bd-752c744476cb service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:43.409053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1244/2468] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:43 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:16:43.635336 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698b0be-04bd-423b-a557-a7f0d2ff3f17 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:43.679012 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:43.729109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698b0be-04bd-423b-a557-a7f0d2ff3f17 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:43.739342 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e49839-70c9-4618-87b4-91becc9bb7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:43.740000 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e49839-70c9-4618-87b4-91becc9bb7ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:43.740278 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e49839-70c9-4618-87b4-91becc9bb7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:43.740543 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e49839-70c9-4618-87b4-91becc9bb7ae admin admin] PATH_INFO: `/v3/users/ef102be9234f4003adb7856b2a1308da` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:43.742219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e49839-70c9-4618-87b4-91becc9bb7ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef102be9234f4003adb7856b2a1308da)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:43.744299 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e49839-70c9-4618-87b4-91becc9bb7ae admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:43.966620 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-26e49839-70c9-4618-87b4-91becc9bb7ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef102be9234f4003adb7856b2a1308da'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:44.028360 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-26e49839-70c9-4618-87b4-91becc9bb7ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef102be9234f4003adb7856b2a1308da'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:44.029034 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-26e49839-70c9-4618-87b4-91becc9bb7ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef102be9234f4003adb7856b2a1308da'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:44.030228 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-26e49839-70c9-4618-87b4-91becc9bb7ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef102be9234f4003adb7856b2a1308da'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:44.039367 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-26e49839-70c9-4618-87b4-91becc9bb7ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef102be9234f4003adb7856b2a1308da'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:44.065106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1225/2469] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:16:43 2022] DELETE /identity/v3/users/ef102be9234f4003adb7856b2a1308da => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:44.069437 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58d462-d882-476f-9713-8b92c7e9c502 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:44.069877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58d462-d882-476f-9713-8b92c7e9c502 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:44.069995 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58d462-d882-476f-9713-8b92c7e9c502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:44.097389 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9b58d462-d882-476f-9713-8b92c7e9c502 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:44.098822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9b58d462-d882-476f-9713-8b92c7e9c502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:44.139313 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e49839-70c9-4618-87b4-91becc9bb7ae admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:44.141762 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-26e49839-70c9-4618-87b4-91becc9bb7ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:44.160436 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:44.191312 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1245/2470] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:44.200089 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58d462-d882-476f-9713-8b92c7e9c502 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:44.202427 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9b58d462-d882-476f-9713-8b92c7e9c502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:44.269687 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e49839-70c9-4618-87b4-91becc9bb7ae admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:44.283636 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d886516-ce70-4178-98f5-df00ab952978 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:44.284526 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d886516-ce70-4178-98f5-df00ab952978 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:44.284735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d886516-ce70-4178-98f5-df00ab952978 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:44.285027 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d886516-ce70-4178-98f5-df00ab952978 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:44.287807 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1d886516-ce70-4178-98f5-df00ab952978 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:44.297330 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58d462-d882-476f-9713-8b92c7e9c502 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:44.302729 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac9d08-ee07-4462-8a08-de0dde18f2bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:44.302983 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d886516-ce70-4178-98f5-df00ab952978 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:44.303821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac9d08-ee07-4462-8a08-de0dde18f2bd service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:44.304326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac9d08-ee07-4462-8a08-de0dde18f2bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:44.304793 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d886516-ce70-4178-98f5-df00ab952978 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:44.305072 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac9d08-ee07-4462-8a08-de0dde18f2bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:44.318921 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac9d08-ee07-4462-8a08-de0dde18f2bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:44.320388 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac9d08-ee07-4462-8a08-de0dde18f2bd service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:44.323432 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1226/2471] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:16:44.332677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1246/2472] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:16:44.976036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d886516-ce70-4178-98f5-df00ab952978 service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:44.977865 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1d886516-ce70-4178-98f5-df00ab952978 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:45.001634 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d886516-ce70-4178-98f5-df00ab952978 service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:45.012939 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f353061-b84e-4f13-8074-fef91e917b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:45.016138 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f353061-b84e-4f13-8074-fef91e917b00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:45.016444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f353061-b84e-4f13-8074-fef91e917b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:45.016733 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f353061-b84e-4f13-8074-fef91e917b00 admin admin] PATH_INFO: `/v3/projects/477a1089a7864481b1f9d2eded4bc057` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:45.019512 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f353061-b84e-4f13-8074-fef91e917b00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=477a1089a7864481b1f9d2eded4bc057)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:45.023622 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f353061-b84e-4f13-8074-fef91e917b00 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:45.127902 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:45.158572 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5f353061-b84e-4f13-8074-fef91e917b00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '477a1089a7864481b1f9d2eded4bc057', 'request_id': 'req-5f353061-b84e-4f13-8074-fef91e917b00'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:45.164984 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5f353061-b84e-4f13-8074-fef91e917b00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '477a1089a7864481b1f9d2eded4bc057', 'request_id': 'req-5f353061-b84e-4f13-8074-fef91e917b00'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:45.286245 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5f353061-b84e-4f13-8074-fef91e917b00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '477a1089a7864481b1f9d2eded4bc057', 'request_id': 'req-5f353061-b84e-4f13-8074-fef91e917b00'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:45.299110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5f353061-b84e-4f13-8074-fef91e917b00 admin admin] The token cache is being invalidate because project 477a1089a7864481b1f9d2eded4bc057 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:16:45.299529 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5f353061-b84e-4f13-8074-fef91e917b00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:45.300272 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5f353061-b84e-4f13-8074-fef91e917b00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:45.304708 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1227/2473] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:16:44 2022] DELETE /identity/v3/projects/477a1089a7864481b1f9d2eded4bc057 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:45.305965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac9d08-ee07-4462-8a08-de0dde18f2bd service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:45.319694 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac9d08-ee07-4462-8a08-de0dde18f2bd service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:45.409156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5aecc2-08a0-48f9-8c16-572a13d167bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:45.409868 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5aecc2-08a0-48f9-8c16-572a13d167bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:45.410326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5aecc2-08a0-48f9-8c16-572a13d167bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:45.410842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5aecc2-08a0-48f9-8c16-572a13d167bf admin admin] PATH_INFO: `/v3/users/dbdb2f11a4e5448bb3f6ee81df3771e0` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:45.412089 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5aecc2-08a0-48f9-8c16-572a13d167bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbdb2f11a4e5448bb3f6ee81df3771e0)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:45.414077 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5aecc2-08a0-48f9-8c16-572a13d167bf admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:45.623903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-fc5aecc2-08a0-48f9-8c16-572a13d167bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbdb2f11a4e5448bb3f6ee81df3771e0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:45.650208 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-fc5aecc2-08a0-48f9-8c16-572a13d167bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbdb2f11a4e5448bb3f6ee81df3771e0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:46.005377 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-fc5aecc2-08a0-48f9-8c16-572a13d167bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbdb2f11a4e5448bb3f6ee81df3771e0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:46.006015 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-fc5aecc2-08a0-48f9-8c16-572a13d167bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbdb2f11a4e5448bb3f6ee81df3771e0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:46.006687 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-fc5aecc2-08a0-48f9-8c16-572a13d167bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbdb2f11a4e5448bb3f6ee81df3771e0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:46.024899 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1247/2474] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:16:45 2022] DELETE /identity/v3/users/dbdb2f11a4e5448bb3f6ee81df3771e0 => generated 0 bytes in 720 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:46.550827 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f353061-b84e-4f13-8074-fef91e917b00 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:46.552644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5f353061-b84e-4f13-8074-fef91e917b00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:46.572917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f353061-b84e-4f13-8074-fef91e917b00 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:46.667800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81986cc-3965-4066-90aa-e2519d0d3bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:46.669274 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81986cc-3965-4066-90aa-e2519d0d3bab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:46.669274 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81986cc-3965-4066-90aa-e2519d0d3bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:46.669274 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81986cc-3965-4066-90aa-e2519d0d3bab admin admin] PATH_INFO: `/v3/projects/dd400459c29647b3b5b9522bb2374354` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:46.672053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81986cc-3965-4066-90aa-e2519d0d3bab admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd400459c29647b3b5b9522bb2374354)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:46.673415 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81986cc-3965-4066-90aa-e2519d0d3bab admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:46.808227 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d81986cc-3965-4066-90aa-e2519d0d3bab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd400459c29647b3b5b9522bb2374354', 'request_id': 'req-d81986cc-3965-4066-90aa-e2519d0d3bab'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:46.816393 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d81986cc-3965-4066-90aa-e2519d0d3bab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd400459c29647b3b5b9522bb2374354', 'request_id': 'req-d81986cc-3965-4066-90aa-e2519d0d3bab'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:46.868316 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d81986cc-3965-4066-90aa-e2519d0d3bab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd400459c29647b3b5b9522bb2374354', 'request_id': 'req-d81986cc-3965-4066-90aa-e2519d0d3bab'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:46.882703 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d81986cc-3965-4066-90aa-e2519d0d3bab admin admin] The token cache is being invalidate because project dd400459c29647b3b5b9522bb2374354 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:16:46.883091 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d81986cc-3965-4066-90aa-e2519d0d3bab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:46.883879 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d81986cc-3965-4066-90aa-e2519d0d3bab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:46.885409 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1228/2475] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:16:46 2022] DELETE /identity/v3/projects/dd400459c29647b3b5b9522bb2374354 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:46.890984 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee451f-8e4a-48db-b29a-5b4415239782 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:46.891528 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee451f-8e4a-48db-b29a-5b4415239782 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:46.891621 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee451f-8e4a-48db-b29a-5b4415239782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:46.901661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:46.912789 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-8bee451f-8e4a-48db-b29a-5b4415239782 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:46.914211 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8bee451f-8e4a-48db-b29a-5b4415239782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:46.998386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1248/2476] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:47.001476 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79537a57-f1b5-4c8a-a8fc-41b1bf7f2ced None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:47.001476 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79537a57-f1b5-4c8a-a8fc-41b1bf7f2ced None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:47.001476 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79537a57-f1b5-4c8a-a8fc-41b1bf7f2ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:47.022604 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-79537a57-f1b5-4c8a-a8fc-41b1bf7f2ced None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:47.024023 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-79537a57-f1b5-4c8a-a8fc-41b1bf7f2ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:47.036255 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1229/2477] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:47.038017 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee451f-8e4a-48db-b29a-5b4415239782 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:47.049404 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee451f-8e4a-48db-b29a-5b4415239782 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:47.054125 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb25136-2d7b-4769-a33c-2d8eebdfa99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:47.055420 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb25136-2d7b-4769-a33c-2d8eebdfa99f admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:47.056181 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb25136-2d7b-4769-a33c-2d8eebdfa99f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:47.056181 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb25136-2d7b-4769-a33c-2d8eebdfa99f admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:47.058728 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb25136-2d7b-4769-a33c-2d8eebdfa99f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:16:47.058728 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb25136-2d7b-4769-a33c-2d8eebdfa99f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:47.058728 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb25136-2d7b-4769-a33c-2d8eebdfa99f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:47.074157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1249/2478] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 13:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:16:47.075323 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79537a57-f1b5-4c8a-a8fc-41b1bf7f2ced None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:47.085863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79537a57-f1b5-4c8a-a8fc-41b1bf7f2ced None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:47.090787 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78957f5-1dec-46ad-a9e8-b2ea80ba56ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:47.091434 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78957f5-1dec-46ad-a9e8-b2ea80ba56ee admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:47.091704 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78957f5-1dec-46ad-a9e8-b2ea80ba56ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:47.091978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78957f5-1dec-46ad-a9e8-b2ea80ba56ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:47.092599 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78957f5-1dec-46ad-a9e8-b2ea80ba56ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:47.095206 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78957f5-1dec-46ad-a9e8-b2ea80ba56ee admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:47.208486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1230/2479] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:16:47 2022] POST /identity/v3/projects => generated 387 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:16:47.212315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb25136-2d7b-4769-a33c-2d8eebdfa99f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:47.223805 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb25136-2d7b-4769-a33c-2d8eebdfa99f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:47.295314 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b8216-a336-4650-9ebb-d5dd94364f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:47.296088 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b8216-a336-4650-9ebb-d5dd94364f2d admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:47.296458 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b8216-a336-4650-9ebb-d5dd94364f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:47.296889 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b8216-a336-4650-9ebb-d5dd94364f2d admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:47.297589 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b8216-a336-4650-9ebb-d5dd94364f2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:47.299729 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b8216-a336-4650-9ebb-d5dd94364f2d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:47.397115 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1250/2480] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:16:47 2022] POST /identity/v3/users => generated 294 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:16:47.401066 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78957f5-1dec-46ad-a9e8-b2ea80ba56ee admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:47.412656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78957f5-1dec-46ad-a9e8-b2ea80ba56ee admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:47.419209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5015a6c-4a69-4265-b56d-08c11a29d72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:47.419209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5015a6c-4a69-4265-b56d-08c11a29d72e admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:47.419209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5015a6c-4a69-4265-b56d-08c11a29d72e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:47.419209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5015a6c-4a69-4265-b56d-08c11a29d72e admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:47.419609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5015a6c-4a69-4265-b56d-08c11a29d72e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:47.421594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5015a6c-4a69-4265-b56d-08c11a29d72e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:47.439595 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b8216-a336-4650-9ebb-d5dd94364f2d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:47.446021 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1231/2481] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:16:47.449709 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b8216-a336-4650-9ebb-d5dd94364f2d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:47.455720 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555e812-193b-4119-a5f0-8a2f1512e3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:47.456798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555e812-193b-4119-a5f0-8a2f1512e3d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:47.456798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555e812-193b-4119-a5f0-8a2f1512e3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:47.456798 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555e812-193b-4119-a5f0-8a2f1512e3d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:47.458221 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8555e812-193b-4119-a5f0-8a2f1512e3d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:47.459466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8555e812-193b-4119-a5f0-8a2f1512e3d0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:47.472312 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1251/2482] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:16:47.476321 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5015a6c-4a69-4265-b56d-08c11a29d72e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:47.483277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:47.489986 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5015a6c-4a69-4265-b56d-08c11a29d72e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:47.495647 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a92aeb-cd52-4ee7-8c73-5b6a20e891f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:47.496554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a92aeb-cd52-4ee7-8c73-5b6a20e891f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:47.497019 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a92aeb-cd52-4ee7-8c73-5b6a20e891f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:47.497475 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a92aeb-cd52-4ee7-8c73-5b6a20e891f5 admin admin] PATH_INFO: `/v3/projects/cc47507f8f2a44f695a2f94b4945e5a6/users/f7892aa74cb94dec811cd4eaf667be89/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:47.536637 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a92aeb-cd52-4ee7-8c73-5b6a20e891f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc47507f8f2a44f695a2f94b4945e5a6, user_id=f7892aa74cb94dec811cd4eaf667be89, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:47.539670 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a92aeb-cd52-4ee7-8c73-5b6a20e891f5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:47.589090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1232/2483] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:16:47 2022] PUT /identity/v3/projects/cc47507f8f2a44f695a2f94b4945e5a6/users/f7892aa74cb94dec811cd4eaf667be89/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:47.601002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41e083-e2ac-4e4d-bd19-9eb9aad5b2b1 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:47.601293 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41e083-e2ac-4e4d-bd19-9eb9aad5b2b1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:47.601570 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41e083-e2ac-4e4d-bd19-9eb9aad5b2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:47.642587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ee41e083-e2ac-4e4d-bd19-9eb9aad5b2b1 None None] MFA Rules not processed for user `f7892aa74cb94dec811cd4eaf667be89`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:47.643788 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ee41e083-e2ac-4e4d-bd19-9eb9aad5b2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:47.730120 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:47.734830 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1252/2484] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:16:47.739249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a92aeb-cd52-4ee7-8c73-5b6a20e891f5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:47.870748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a92aeb-cd52-4ee7-8c73-5b6a20e891f5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:47.891108 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f0f8c-43be-4bfd-8135-efed97b969b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:47.891815 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f0f8c-43be-4bfd-8135-efed97b969b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:47.892095 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f0f8c-43be-4bfd-8135-efed97b969b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:47.893013 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f0f8c-43be-4bfd-8135-efed97b969b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:47.893013 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f0f8c-43be-4bfd-8135-efed97b969b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:47.896148 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f0f8c-43be-4bfd-8135-efed97b969b4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:48.002960 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1233/2485] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:16:47 2022] POST /identity/v3/projects => generated 387 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:16:48.006782 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41e083-e2ac-4e4d-bd19-9eb9aad5b2b1 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:48.031478 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41e083-e2ac-4e4d-bd19-9eb9aad5b2b1 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:48.112936 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fcc12-2284-4031-9946-82460e4eff5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:48.113703 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fcc12-2284-4031-9946-82460e4eff5c admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:48.114081 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fcc12-2284-4031-9946-82460e4eff5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:48.114610 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fcc12-2284-4031-9946-82460e4eff5c admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:48.115380 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195fcc12-2284-4031-9946-82460e4eff5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:48.117442 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195fcc12-2284-4031-9946-82460e4eff5c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:48.193732 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1253/2486] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:16:48 2022] POST /identity/v3/users => generated 300 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:16:48.196299 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f0f8c-43be-4bfd-8135-efed97b969b4 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:48.208564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f0f8c-43be-4bfd-8135-efed97b969b4 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:48.216699 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5898c-9f01-47cd-82f6-67a210baa0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:48.218774 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5898c-9f01-47cd-82f6-67a210baa0a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:48.218774 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5898c-9f01-47cd-82f6-67a210baa0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:48.218774 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5898c-9f01-47cd-82f6-67a210baa0a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:48.219352 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5898c-9f01-47cd-82f6-67a210baa0a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:48.221106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5898c-9f01-47cd-82f6-67a210baa0a5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:48.239225 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1234/2487] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:16:48.241644 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fcc12-2284-4031-9946-82460e4eff5c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:48.253041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fcc12-2284-4031-9946-82460e4eff5c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:48.258264 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dcf604-0fdc-4a3e-a41a-ea0a17129c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:48.258926 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcf604-0fdc-4a3e-a41a-ea0a17129c8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:48.259231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcf604-0fdc-4a3e-a41a-ea0a17129c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:48.259514 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcf604-0fdc-4a3e-a41a-ea0a17129c8e admin admin] PATH_INFO: `/v3/projects/2ca698a6fa5b4c67848425e03b1f2a00/users/c6880e8336f0437b82c4d4485388fcaf/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:48.281180 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dcf604-0fdc-4a3e-a41a-ea0a17129c8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ca698a6fa5b4c67848425e03b1f2a00, user_id=c6880e8336f0437b82c4d4485388fcaf, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:48.284685 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dcf604-0fdc-4a3e-a41a-ea0a17129c8e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:48.349090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1254/2488] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:16:48 2022] PUT /identity/v3/projects/2ca698a6fa5b4c67848425e03b1f2a00/users/c6880e8336f0437b82c4d4485388fcaf/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:16:48.353027 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5898c-9f01-47cd-82f6-67a210baa0a5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:48.367145 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5898c-9f01-47cd-82f6-67a210baa0a5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:48.456179 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874e4ede-2306-42dd-a63a-8bf0adb9d43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:48.457076 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e4ede-2306-42dd-a63a-8bf0adb9d43c admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:48.457516 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e4ede-2306-42dd-a63a-8bf0adb9d43c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:48.457802 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e4ede-2306-42dd-a63a-8bf0adb9d43c admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:48.458442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874e4ede-2306-42dd-a63a-8bf0adb9d43c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:48.460084 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874e4ede-2306-42dd-a63a-8bf0adb9d43c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:48.475792 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1235/2489] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:16:48.477994 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dcf604-0fdc-4a3e-a41a-ea0a17129c8e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:48.491696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dcf604-0fdc-4a3e-a41a-ea0a17129c8e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:48.496906 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf99901c-61f9-4c33-a05b-c1cf2c0483d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:48.498161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99901c-61f9-4c33-a05b-c1cf2c0483d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:48.498441 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99901c-61f9-4c33-a05b-c1cf2c0483d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:48.498754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99901c-61f9-4c33-a05b-c1cf2c0483d8 admin admin] PATH_INFO: `/v3/projects/2ca698a6fa5b4c67848425e03b1f2a00/users/c6880e8336f0437b82c4d4485388fcaf/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:48.501423 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf99901c-61f9-4c33-a05b-c1cf2c0483d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ca698a6fa5b4c67848425e03b1f2a00, user_id=c6880e8336f0437b82c4d4485388fcaf, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:48.504431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf99901c-61f9-4c33-a05b-c1cf2c0483d8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:48.518636 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1255/2490] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:16:48 2022] PUT /identity/v3/projects/2ca698a6fa5b4c67848425e03b1f2a00/users/c6880e8336f0437b82c4d4485388fcaf/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:48.529526 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96071ccd-0174-48d3-9415-c1ea2e36eb09 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:48.529966 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96071ccd-0174-48d3-9415-c1ea2e36eb09 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:48.530454 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96071ccd-0174-48d3-9415-c1ea2e36eb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:48.565392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-96071ccd-0174-48d3-9415-c1ea2e36eb09 None None] MFA Rules not processed for user `c6880e8336f0437b82c4d4485388fcaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:48.569307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-96071ccd-0174-48d3-9415-c1ea2e36eb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:48.627620 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:48.667277 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1236/2491] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:48.674788 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf99901c-61f9-4c33-a05b-c1cf2c0483d8 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:48.676815 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cf99901c-61f9-4c33-a05b-c1cf2c0483d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:48.755121 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf99901c-61f9-4c33-a05b-c1cf2c0483d8 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:48.760193 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725bbfa1-7f1e-43b2-be2f-d4acc2dedcbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:48.761088 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725bbfa1-7f1e-43b2-be2f-d4acc2dedcbb service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:48.761439 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725bbfa1-7f1e-43b2-be2f-d4acc2dedcbb service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:48.761720 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725bbfa1-7f1e-43b2-be2f-d4acc2dedcbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:48.773075 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725bbfa1-7f1e-43b2-be2f-d4acc2dedcbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:48.774817 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725bbfa1-7f1e-43b2-be2f-d4acc2dedcbb service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:48.787471 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1256/2492] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:16:48.819886 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96071ccd-0174-48d3-9415-c1ea2e36eb09 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:48.928913 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96071ccd-0174-48d3-9415-c1ea2e36eb09 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:48.958087 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982fe55c-a617-4123-95e9-07df8ef339ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:48.959388 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fe55c-a617-4123-95e9-07df8ef339ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:48.960398 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fe55c-a617-4123-95e9-07df8ef339ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:48.961007 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fe55c-a617-4123-95e9-07df8ef339ba admin admin] PATH_INFO: `/v3/users/f7892aa74cb94dec811cd4eaf667be89` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:48.966370 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982fe55c-a617-4123-95e9-07df8ef339ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7892aa74cb94dec811cd4eaf667be89)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:48.971994 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982fe55c-a617-4123-95e9-07df8ef339ba admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:49.343262 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-982fe55c-a617-4123-95e9-07df8ef339ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7892aa74cb94dec811cd4eaf667be89'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:49.422863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-982fe55c-a617-4123-95e9-07df8ef339ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7892aa74cb94dec811cd4eaf667be89'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:49.423697 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-982fe55c-a617-4123-95e9-07df8ef339ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7892aa74cb94dec811cd4eaf667be89'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:49.425295 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-982fe55c-a617-4123-95e9-07df8ef339ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7892aa74cb94dec811cd4eaf667be89'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:49.438472 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-982fe55c-a617-4123-95e9-07df8ef339ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7892aa74cb94dec811cd4eaf667be89'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:49.462175 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1237/2493] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:16:48 2022] DELETE /identity/v3/users/f7892aa74cb94dec811cd4eaf667be89 => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:49.466300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a924b-335a-4018-9fea-29e70014f760 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:49.466779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a924b-335a-4018-9fea-29e70014f760 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:49.466903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a924b-335a-4018-9fea-29e70014f760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:49.488123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-1c8a924b-335a-4018-9fea-29e70014f760 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:49.489362 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1c8a924b-335a-4018-9fea-29e70014f760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:49.540996 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:49.559351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1257/2494] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:49.564126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982fe55c-a617-4123-95e9-07df8ef339ba admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:49.567418 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-982fe55c-a617-4123-95e9-07df8ef339ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:49.636945 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a924b-335a-4018-9fea-29e70014f760 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:49.640782 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1c8a924b-335a-4018-9fea-29e70014f760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:49.707466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982fe55c-a617-4123-95e9-07df8ef339ba admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:49.713796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b0163e-a415-48d6-a5ce-70d4d3b882a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:49.714694 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0163e-a415-48d6-a5ce-70d4d3b882a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:49.715073 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0163e-a415-48d6-a5ce-70d4d3b882a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:49.715617 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0163e-a415-48d6-a5ce-70d4d3b882a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:49.737729 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b0163e-a415-48d6-a5ce-70d4d3b882a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:49.741017 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b0163e-a415-48d6-a5ce-70d4d3b882a4 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:49.752144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:49.759322 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a924b-335a-4018-9fea-29e70014f760 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:49.763005 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1238/2495] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:16:49.763450 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337bfa29-7420-4ac3-b54a-281b69235316 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:49.764461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bfa29-7420-4ac3-b54a-281b69235316 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:49.764815 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bfa29-7420-4ac3-b54a-281b69235316 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:49.765736 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bfa29-7420-4ac3-b54a-281b69235316 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:49.767900 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-337bfa29-7420-4ac3-b54a-281b69235316 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:49.883217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337bfa29-7420-4ac3-b54a-281b69235316 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:49.886581 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337bfa29-7420-4ac3-b54a-281b69235316 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:49.904442 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1258/2496] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:49 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:16:50.305199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b0163e-a415-48d6-a5ce-70d4d3b882a4 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:50.309220 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-97b0163e-a415-48d6-a5ce-70d4d3b882a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:50.326545 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b0163e-a415-48d6-a5ce-70d4d3b882a4 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:50.329522 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897479fb-2868-47e3-a05c-6a9e13598d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:50.330335 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897479fb-2868-47e3-a05c-6a9e13598d6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:50.330722 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897479fb-2868-47e3-a05c-6a9e13598d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:50.331060 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897479fb-2868-47e3-a05c-6a9e13598d6d admin admin] PATH_INFO: `/v3/projects/cc47507f8f2a44f695a2f94b4945e5a6` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:50.332357 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897479fb-2868-47e3-a05c-6a9e13598d6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc47507f8f2a44f695a2f94b4945e5a6)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:50.334416 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897479fb-2868-47e3-a05c-6a9e13598d6d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:50.429892 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-897479fb-2868-47e3-a05c-6a9e13598d6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc47507f8f2a44f695a2f94b4945e5a6', 'request_id': 'req-897479fb-2868-47e3-a05c-6a9e13598d6d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:50.435623 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-897479fb-2868-47e3-a05c-6a9e13598d6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc47507f8f2a44f695a2f94b4945e5a6', 'request_id': 'req-897479fb-2868-47e3-a05c-6a9e13598d6d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:50.519356 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-897479fb-2868-47e3-a05c-6a9e13598d6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc47507f8f2a44f695a2f94b4945e5a6', 'request_id': 'req-897479fb-2868-47e3-a05c-6a9e13598d6d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:50.531769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-897479fb-2868-47e3-a05c-6a9e13598d6d admin admin] The token cache is being invalidate because project cc47507f8f2a44f695a2f94b4945e5a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:16:50.532063 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-897479fb-2868-47e3-a05c-6a9e13598d6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:50.532719 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-897479fb-2868-47e3-a05c-6a9e13598d6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:50.534190 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1239/2497] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:16:50 2022] DELETE /identity/v3/projects/cc47507f8f2a44f695a2f94b4945e5a6 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:50.538231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337bfa29-7420-4ac3-b54a-281b69235316 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:50.538692 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3945af-94c4-4704-b689-b03e741c3915 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:50.538984 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3945af-94c4-4704-b689-b03e741c3915 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:50.539435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3945af-94c4-4704-b689-b03e741c3915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:50.549869 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337bfa29-7420-4ac3-b54a-281b69235316 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:50.562841 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1c3945af-94c4-4704-b689-b03e741c3915 None None] MFA Rules not processed for user `c025957730c04df5980279aaef8d0807`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:50.564351 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1c3945af-94c4-4704-b689-b03e741c3915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:50.613894 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a35d4-26de-48a9-83e8-5b304f5b15c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:50.614589 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a35d4-26de-48a9-83e8-5b304f5b15c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:50.614911 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a35d4-26de-48a9-83e8-5b304f5b15c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:50.615286 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a35d4-26de-48a9-83e8-5b304f5b15c7 admin admin] PATH_INFO: `/v3/users/c6880e8336f0437b82c4d4485388fcaf` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:50.616574 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a35d4-26de-48a9-83e8-5b304f5b15c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6880e8336f0437b82c4d4485388fcaf)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:50.618542 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a35d4-26de-48a9-83e8-5b304f5b15c7 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:50.629987 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1240/2498] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:16:50.755987 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ca8a35d4-26de-48a9-83e8-5b304f5b15c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6880e8336f0437b82c4d4485388fcaf'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:50.757168 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:50.763455 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ca8a35d4-26de-48a9-83e8-5b304f5b15c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6880e8336f0437b82c4d4485388fcaf'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:50.822983 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ca8a35d4-26de-48a9-83e8-5b304f5b15c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6880e8336f0437b82c4d4485388fcaf'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:50.824099 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ca8a35d4-26de-48a9-83e8-5b304f5b15c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6880e8336f0437b82c4d4485388fcaf'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:50.824731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ca8a35d4-26de-48a9-83e8-5b304f5b15c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6880e8336f0437b82c4d4485388fcaf'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:50.847914 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1259/2499] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:16:50 2022] DELETE /identity/v3/users/c6880e8336f0437b82c4d4485388fcaf => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:51.368493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3945af-94c4-4704-b689-b03e741c3915 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:51.370326 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1c3945af-94c4-4704-b689-b03e741c3915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:51.388818 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3945af-94c4-4704-b689-b03e741c3915 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:51.470251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0531028-85d7-40a0-9169-13834b122319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:51.470251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0531028-85d7-40a0-9169-13834b122319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:51.470251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0531028-85d7-40a0-9169-13834b122319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:51.470251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0531028-85d7-40a0-9169-13834b122319 admin admin] PATH_INFO: `/v3/projects/2ca698a6fa5b4c67848425e03b1f2a00` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:51.470728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0531028-85d7-40a0-9169-13834b122319 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ca698a6fa5b4c67848425e03b1f2a00)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:51.470728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0531028-85d7-40a0-9169-13834b122319 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:51.556746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:51.571771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f0531028-85d7-40a0-9169-13834b122319 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ca698a6fa5b4c67848425e03b1f2a00', 'request_id': 'req-f0531028-85d7-40a0-9169-13834b122319'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:51.577826 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f0531028-85d7-40a0-9169-13834b122319 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ca698a6fa5b4c67848425e03b1f2a00', 'request_id': 'req-f0531028-85d7-40a0-9169-13834b122319'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:51.657390 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f0531028-85d7-40a0-9169-13834b122319 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ca698a6fa5b4c67848425e03b1f2a00', 'request_id': 'req-f0531028-85d7-40a0-9169-13834b122319'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:51.669098 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f0531028-85d7-40a0-9169-13834b122319 admin admin] The token cache is being invalidate because project 2ca698a6fa5b4c67848425e03b1f2a00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:16:51.669388 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f0531028-85d7-40a0-9169-13834b122319 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:51.670026 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f0531028-85d7-40a0-9169-13834b122319 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:51.671618 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1241/2500] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:16:51 2022] DELETE /identity/v3/projects/2ca698a6fa5b4c67848425e03b1f2a00 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:51.676676 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f264b56-eef9-4c92-86b8-53650cf9139c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:51.677160 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f264b56-eef9-4c92-86b8-53650cf9139c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:51.677248 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f264b56-eef9-4c92-86b8-53650cf9139c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:51.697741 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9f264b56-eef9-4c92-86b8-53650cf9139c None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:51.699047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9f264b56-eef9-4c92-86b8-53650cf9139c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:51.767061 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1260/2501] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:51.774924 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849eee8-4d50-4f53-aadd-d9ea6b96bb94 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:51.775285 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849eee8-4d50-4f53-aadd-d9ea6b96bb94 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:51.775614 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849eee8-4d50-4f53-aadd-d9ea6b96bb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:51.795684 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7849eee8-4d50-4f53-aadd-d9ea6b96bb94 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:51.796904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7849eee8-4d50-4f53-aadd-d9ea6b96bb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:51.804871 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1242/2502] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:51.809104 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f264b56-eef9-4c92-86b8-53650cf9139c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:51.819378 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f264b56-eef9-4c92-86b8-53650cf9139c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:51.824427 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c28fb-f0fe-40c8-ac79-8daa375a90a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:51.825127 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c28fb-f0fe-40c8-ac79-8daa375a90a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:51.825403 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c28fb-f0fe-40c8-ac79-8daa375a90a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:51.825689 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c28fb-f0fe-40c8-ac79-8daa375a90a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:51.826414 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c28fb-f0fe-40c8-ac79-8daa375a90a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:16:51.826711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c28fb-f0fe-40c8-ac79-8daa375a90a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:51.828864 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c28fb-f0fe-40c8-ac79-8daa375a90a5 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:51.840429 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1261/2503] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 13:16:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:16:51.845969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849eee8-4d50-4f53-aadd-d9ea6b96bb94 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:51.855356 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849eee8-4d50-4f53-aadd-d9ea6b96bb94 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:51.860607 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c94f0-1975-4ace-913f-3f423fb74c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:51.861198 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c94f0-1975-4ace-913f-3f423fb74c9c admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:51.861472 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c94f0-1975-4ace-913f-3f423fb74c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:51.862017 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c94f0-1975-4ace-913f-3f423fb74c9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:51.862639 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1c94f0-1975-4ace-913f-3f423fb74c9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:51.864551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1c94f0-1975-4ace-913f-3f423fb74c9c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:51.926509 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1243/2504] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:16:51 2022] POST /identity/v3/projects => generated 385 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:16:51.930926 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c28fb-f0fe-40c8-ac79-8daa375a90a5 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:51.941502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c28fb-f0fe-40c8-ac79-8daa375a90a5 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:51.997217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c2cb0-a432-46db-8709-a9510c9db460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:51.997840 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c2cb0-a432-46db-8709-a9510c9db460 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:51.998139 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c2cb0-a432-46db-8709-a9510c9db460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:51.998385 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c2cb0-a432-46db-8709-a9510c9db460 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:51.999003 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550c2cb0-a432-46db-8709-a9510c9db460 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:52.001973 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550c2cb0-a432-46db-8709-a9510c9db460 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:52.102473 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1262/2505] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:16:51 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:16:52.106493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c94f0-1975-4ace-913f-3f423fb74c9c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:52.117280 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c94f0-1975-4ace-913f-3f423fb74c9c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:52.122587 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b4c2a-8aad-4dae-95dd-2832df2fc22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:52.123253 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b4c2a-8aad-4dae-95dd-2832df2fc22a admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:52.123580 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b4c2a-8aad-4dae-95dd-2832df2fc22a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:52.123869 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b4c2a-8aad-4dae-95dd-2832df2fc22a admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:52.125177 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b4c2a-8aad-4dae-95dd-2832df2fc22a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:52.126795 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b4c2a-8aad-4dae-95dd-2832df2fc22a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:52.143567 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1244/2506] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:16:52.143688 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c2cb0-a432-46db-8709-a9510c9db460 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:52.153106 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c2cb0-a432-46db-8709-a9510c9db460 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:52.157990 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e213c-63ba-4a1c-a87b-ada5489d58ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:52.158578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e213c-63ba-4a1c-a87b-ada5489d58ec admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:52.158868 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e213c-63ba-4a1c-a87b-ada5489d58ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:52.159135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e213c-63ba-4a1c-a87b-ada5489d58ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:52.159856 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e213c-63ba-4a1c-a87b-ada5489d58ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:52.161463 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e213c-63ba-4a1c-a87b-ada5489d58ec admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:52.172307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1263/2507] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:16:52.176363 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b4c2a-8aad-4dae-95dd-2832df2fc22a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:52.186420 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b4c2a-8aad-4dae-95dd-2832df2fc22a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:52.192161 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d0151-f2ca-4dad-a345-54ef304739af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:52.192816 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d0151-f2ca-4dad-a345-54ef304739af admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:52.193100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d0151-f2ca-4dad-a345-54ef304739af admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:52.193380 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d0151-f2ca-4dad-a345-54ef304739af admin admin] PATH_INFO: `/v3/projects/cd4485ee86394e788c2a7f2a736a18ce/users/45dda99013ec43f083d2676aa548c281/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:52.209260 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025d0151-f2ca-4dad-a345-54ef304739af admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd4485ee86394e788c2a7f2a736a18ce, user_id=45dda99013ec43f083d2676aa548c281, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:52.211943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025d0151-f2ca-4dad-a345-54ef304739af admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:52.297605 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1245/2508] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:16:52 2022] PUT /identity/v3/projects/cd4485ee86394e788c2a7f2a736a18ce/users/45dda99013ec43f083d2676aa548c281/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:52.308928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f90222-0e70-4eb6-9446-57d4ed7547d4 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:52.309209 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f90222-0e70-4eb6-9446-57d4ed7547d4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:52.309482 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f90222-0e70-4eb6-9446-57d4ed7547d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:52.338468 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-51f90222-0e70-4eb6-9446-57d4ed7547d4 None None] MFA Rules not processed for user `45dda99013ec43f083d2676aa548c281`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:52.339692 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-51f90222-0e70-4eb6-9446-57d4ed7547d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:52.358337 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:52.406720 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1264/2509] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:16:52.417862 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d0151-f2ca-4dad-a345-54ef304739af admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:52.419664 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-025d0151-f2ca-4dad-a345-54ef304739af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:52.478530 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d0151-f2ca-4dad-a345-54ef304739af admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:52.482199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b044e906-9b7b-4586-88eb-23be59612c84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:52.482808 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044e906-9b7b-4586-88eb-23be59612c84 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:52.483096 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044e906-9b7b-4586-88eb-23be59612c84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:52.483419 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044e906-9b7b-4586-88eb-23be59612c84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:52.492666 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b044e906-9b7b-4586-88eb-23be59612c84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:52.494110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b044e906-9b7b-4586-88eb-23be59612c84 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:52.503269 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1246/2510] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:16:52.535602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f90222-0e70-4eb6-9446-57d4ed7547d4 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:52.598578 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f90222-0e70-4eb6-9446-57d4ed7547d4 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:52.603073 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf18fb-8c32-4feb-8f68-88d3eddd3b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:52.603862 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf18fb-8c32-4feb-8f68-88d3eddd3b07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:52.604151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf18fb-8c32-4feb-8f68-88d3eddd3b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:52.604421 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf18fb-8c32-4feb-8f68-88d3eddd3b07 admin admin] PATH_INFO: `/v3/users/45dda99013ec43f083d2676aa548c281` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:52.605538 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bf18fb-8c32-4feb-8f68-88d3eddd3b07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45dda99013ec43f083d2676aa548c281)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:52.607463 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bf18fb-8c32-4feb-8f68-88d3eddd3b07 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:52.813296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-16bf18fb-8c32-4feb-8f68-88d3eddd3b07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45dda99013ec43f083d2676aa548c281'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:52.820095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-16bf18fb-8c32-4feb-8f68-88d3eddd3b07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45dda99013ec43f083d2676aa548c281'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:52.873518 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-16bf18fb-8c32-4feb-8f68-88d3eddd3b07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45dda99013ec43f083d2676aa548c281'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:52.874623 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-16bf18fb-8c32-4feb-8f68-88d3eddd3b07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45dda99013ec43f083d2676aa548c281'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:52.875264 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-16bf18fb-8c32-4feb-8f68-88d3eddd3b07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45dda99013ec43f083d2676aa548c281'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:52.893564 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1265/2511] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:16:52 2022] DELETE /identity/v3/users/45dda99013ec43f083d2676aa548c281 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:52.898151 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be8d2b-31ea-4744-8316-c6e5591d2e02 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:52.898435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be8d2b-31ea-4744-8316-c6e5591d2e02 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:52.898706 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be8d2b-31ea-4744-8316-c6e5591d2e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:52.917389 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c7be8d2b-31ea-4744-8316-c6e5591d2e02 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:52.918658 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c7be8d2b-31ea-4744-8316-c6e5591d2e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:52.967094 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:52.978458 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1247/2512] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:52.988531 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf18fb-8c32-4feb-8f68-88d3eddd3b07 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:52.989928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-16bf18fb-8c32-4feb-8f68-88d3eddd3b07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:53.034596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:53.047865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf18fb-8c32-4feb-8f68-88d3eddd3b07 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:53.051771 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d47aaa5-1c0f-4918-b3dc-34329c58a0bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:53.052371 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47aaa5-1c0f-4918-b3dc-34329c58a0bc service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:53.052648 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47aaa5-1c0f-4918-b3dc-34329c58a0bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:53.052919 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47aaa5-1c0f-4918-b3dc-34329c58a0bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:53.062976 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d47aaa5-1c0f-4918-b3dc-34329c58a0bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:53.067505 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d47aaa5-1c0f-4918-b3dc-34329c58a0bc service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:53.073824 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1266/2513] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:16:53.470221 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be8d2b-31ea-4744-8316-c6e5591d2e02 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:53.474725 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c7be8d2b-31ea-4744-8316-c6e5591d2e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:53.486360 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be8d2b-31ea-4744-8316-c6e5591d2e02 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:53.491857 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c4560-ab2e-4631-ab7c-7a8fb7e31b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:53.492554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c4560-ab2e-4631-ab7c-7a8fb7e31b81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:53.492784 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c4560-ab2e-4631-ab7c-7a8fb7e31b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:53.493064 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c4560-ab2e-4631-ab7c-7a8fb7e31b81 admin admin] PATH_INFO: `/v3/projects/cd4485ee86394e788c2a7f2a736a18ce` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:53.494743 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c4560-ab2e-4631-ab7c-7a8fb7e31b81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd4485ee86394e788c2a7f2a736a18ce)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:53.496680 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c4560-ab2e-4631-ab7c-7a8fb7e31b81 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:53.589650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4e8c4560-ab2e-4631-ab7c-7a8fb7e31b81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd4485ee86394e788c2a7f2a736a18ce', 'request_id': 'req-4e8c4560-ab2e-4631-ab7c-7a8fb7e31b81'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:53.595566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4e8c4560-ab2e-4631-ab7c-7a8fb7e31b81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd4485ee86394e788c2a7f2a736a18ce', 'request_id': 'req-4e8c4560-ab2e-4631-ab7c-7a8fb7e31b81'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:53.696440 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4e8c4560-ab2e-4631-ab7c-7a8fb7e31b81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd4485ee86394e788c2a7f2a736a18ce', 'request_id': 'req-4e8c4560-ab2e-4631-ab7c-7a8fb7e31b81'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:53.708801 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4e8c4560-ab2e-4631-ab7c-7a8fb7e31b81 admin admin] The token cache is being invalidate because project cd4485ee86394e788c2a7f2a736a18ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:16:53.709090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4e8c4560-ab2e-4631-ab7c-7a8fb7e31b81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:53.709740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4e8c4560-ab2e-4631-ab7c-7a8fb7e31b81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:53.711265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1248/2514] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:16:53 2022] DELETE /identity/v3/projects/cd4485ee86394e788c2a7f2a736a18ce => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:53.716874 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc7f4a-f25e-48c5-a767-a679fcfb1ee9 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:53.717162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc7f4a-f25e-48c5-a767-a679fcfb1ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:53.717521 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc7f4a-f25e-48c5-a767-a679fcfb1ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:53.737821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-66fc7f4a-f25e-48c5-a767-a679fcfb1ee9 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:53.739073 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-66fc7f4a-f25e-48c5-a767-a679fcfb1ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:53.800742 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1267/2515] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:53.808412 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e4239-4d0d-4dd6-b94e-eb1f2a03c0c0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:53.808700 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e4239-4d0d-4dd6-b94e-eb1f2a03c0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:53.808976 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e4239-4d0d-4dd6-b94e-eb1f2a03c0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:53.829304 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-493e4239-4d0d-4dd6-b94e-eb1f2a03c0c0 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:53.830516 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-493e4239-4d0d-4dd6-b94e-eb1f2a03c0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:53.843072 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1249/2516] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:53.843270 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc7f4a-f25e-48c5-a767-a679fcfb1ee9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:53.853240 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc7f4a-f25e-48c5-a767-a679fcfb1ee9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:53.858378 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2baf546-761c-49e6-9278-96a4ca5a1c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:53.859625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2baf546-761c-49e6-9278-96a4ca5a1c69 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:53.859887 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2baf546-761c-49e6-9278-96a4ca5a1c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:53.860161 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2baf546-761c-49e6-9278-96a4ca5a1c69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:53.861297 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2baf546-761c-49e6-9278-96a4ca5a1c69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:16:53.861921 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2baf546-761c-49e6-9278-96a4ca5a1c69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:53.863516 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2baf546-761c-49e6-9278-96a4ca5a1c69 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:53.876388 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1268/2517] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 13:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:16:53.880983 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e4239-4d0d-4dd6-b94e-eb1f2a03c0c0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:53.890563 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e4239-4d0d-4dd6-b94e-eb1f2a03c0c0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:53.895256 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4c7c8-014e-4a83-a0c4-038709ffeed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:53.895857 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4c7c8-014e-4a83-a0c4-038709ffeed8 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:53.896140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4c7c8-014e-4a83-a0c4-038709ffeed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:53.896415 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4c7c8-014e-4a83-a0c4-038709ffeed8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:53.897044 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb4c7c8-014e-4a83-a0c4-038709ffeed8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:53.898932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb4c7c8-014e-4a83-a0c4-038709ffeed8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:53.978130 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1250/2518] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:16:53 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:16:53.982277 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2baf546-761c-49e6-9278-96a4ca5a1c69 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:53.995310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2baf546-761c-49e6-9278-96a4ca5a1c69 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:54.054639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696e65f-d3a3-47ed-8f6a-11dc72cbceb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:54.055279 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696e65f-d3a3-47ed-8f6a-11dc72cbceb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:54.055564 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696e65f-d3a3-47ed-8f6a-11dc72cbceb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:54.055838 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696e65f-d3a3-47ed-8f6a-11dc72cbceb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:54.056453 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e696e65f-d3a3-47ed-8f6a-11dc72cbceb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:54.058355 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e696e65f-d3a3-47ed-8f6a-11dc72cbceb3 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:54.153927 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1269/2519] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:16:53 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:16:54.158025 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4c7c8-014e-4a83-a0c4-038709ffeed8 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:54.168735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4c7c8-014e-4a83-a0c4-038709ffeed8 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:54.173881 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721f265-c61e-4921-a27e-349c16edaaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:54.174504 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721f265-c61e-4921-a27e-349c16edaaaf admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:54.174757 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721f265-c61e-4921-a27e-349c16edaaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:54.175031 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721f265-c61e-4921-a27e-349c16edaaaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:54.175717 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7721f265-c61e-4921-a27e-349c16edaaaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:54.180457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7721f265-c61e-4921-a27e-349c16edaaaf admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:54.188110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1251/2520] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:16:54.192012 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696e65f-d3a3-47ed-8f6a-11dc72cbceb3 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:54.202002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696e65f-d3a3-47ed-8f6a-11dc72cbceb3 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:54.207074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe20dd-29c7-4536-b4e9-ef529ccb413e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:54.207694 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe20dd-29c7-4536-b4e9-ef529ccb413e admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:54.209167 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe20dd-29c7-4536-b4e9-ef529ccb413e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:54.209167 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe20dd-29c7-4536-b4e9-ef529ccb413e admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:54.209167 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe20dd-29c7-4536-b4e9-ef529ccb413e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:54.210449 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe20dd-29c7-4536-b4e9-ef529ccb413e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:54.222843 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1270/2521] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:16:54.226806 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721f265-c61e-4921-a27e-349c16edaaaf admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:54.237727 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721f265-c61e-4921-a27e-349c16edaaaf admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:54.242657 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c7496d-eb34-4bdf-a261-fb5a2029ef5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:54.243355 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7496d-eb34-4bdf-a261-fb5a2029ef5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:54.243634 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7496d-eb34-4bdf-a261-fb5a2029ef5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:54.243907 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7496d-eb34-4bdf-a261-fb5a2029ef5e admin admin] PATH_INFO: `/v3/projects/7e6487713f214e96afb9a543a9d94430/users/11cd7f17ba014170935f1c038cca0eee/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:54.264331 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c7496d-eb34-4bdf-a261-fb5a2029ef5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e6487713f214e96afb9a543a9d94430, user_id=11cd7f17ba014170935f1c038cca0eee, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:54.267422 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c7496d-eb34-4bdf-a261-fb5a2029ef5e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:54.319761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1252/2522] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:16:54 2022] PUT /identity/v3/projects/7e6487713f214e96afb9a543a9d94430/users/11cd7f17ba014170935f1c038cca0eee/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:54.334732 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91ad4e-4e73-42cf-bda3-908960185335 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:54.335013 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91ad4e-4e73-42cf-bda3-908960185335 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:54.335340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91ad4e-4e73-42cf-bda3-908960185335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:54.364190 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6c91ad4e-4e73-42cf-bda3-908960185335 None None] MFA Rules not processed for user `11cd7f17ba014170935f1c038cca0eee`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:54.365403 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6c91ad4e-4e73-42cf-bda3-908960185335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:54.403559 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:54.431002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1271/2523] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:54.441027 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c7496d-eb34-4bdf-a261-fb5a2029ef5e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:54.443256 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b6c7496d-eb34-4bdf-a261-fb5a2029ef5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:54.505444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c7496d-eb34-4bdf-a261-fb5a2029ef5e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:54.509278 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0365a9c2-0e93-4c54-beb8-dcfd83d0e57e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:54.509887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365a9c2-0e93-4c54-beb8-dcfd83d0e57e service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:54.510168 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365a9c2-0e93-4c54-beb8-dcfd83d0e57e service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:54.510443 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365a9c2-0e93-4c54-beb8-dcfd83d0e57e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:54.520406 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0365a9c2-0e93-4c54-beb8-dcfd83d0e57e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:54.521485 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0365a9c2-0e93-4c54-beb8-dcfd83d0e57e service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:54.531104 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1253/2524] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:16:55.081604 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b4e53-7e5a-4011-a983-e719e999b256 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:55.082105 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b4e53-7e5a-4011-a983-e719e999b256 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:55.082220 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b4e53-7e5a-4011-a983-e719e999b256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:55.101541 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-758b4e53-7e5a-4011-a983-e719e999b256 None None] MFA Rules not processed for user `e42059e9af9842548ea08dcd39ff0b6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:55.103056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-758b4e53-7e5a-4011-a983-e719e999b256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:55.181452 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1272/2525] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 10 13:16:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:55.195049 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0365a9c2-0e93-4c54-beb8-dcfd83d0e57e service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:55.196567 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0365a9c2-0e93-4c54-beb8-dcfd83d0e57e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:55.217559 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:55.275120 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0365a9c2-0e93-4c54-beb8-dcfd83d0e57e service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:55.282876 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e796e-d883-4965-9e49-4324078bd81b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59395bfa505c4540b3957deee78eb698', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:55.284920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e796e-d883-4965-9e49-4324078bd81b service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:55.285392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e796e-d883-4965-9e49-4324078bd81b service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:55.285871 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e796e-d883-4965-9e49-4324078bd81b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:55.299305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e796e-d883-4965-9e49-4324078bd81b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:55.301608 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e796e-d883-4965-9e49-4324078bd81b service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:55.328978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1254/2526] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 13:16:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:16:57.770949 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b4e53-7e5a-4011-a983-e719e999b256 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:57.841522 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b4e53-7e5a-4011-a983-e719e999b256 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:57.846815 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8397c9e-0f6c-42d8-9fb9-19214add9e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:57.847553 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8397c9e-0f6c-42d8-9fb9-19214add9e81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:57.847776 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8397c9e-0f6c-42d8-9fb9-19214add9e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:57.848054 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8397c9e-0f6c-42d8-9fb9-19214add9e81 admin admin] PATH_INFO: `/v3/users/11cd7f17ba014170935f1c038cca0eee` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:57.849600 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8397c9e-0f6c-42d8-9fb9-19214add9e81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11cd7f17ba014170935f1c038cca0eee)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:57.851570 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8397c9e-0f6c-42d8-9fb9-19214add9e81 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:58.018447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c8397c9e-0f6c-42d8-9fb9-19214add9e81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11cd7f17ba014170935f1c038cca0eee'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:58.025204 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c8397c9e-0f6c-42d8-9fb9-19214add9e81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11cd7f17ba014170935f1c038cca0eee'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:58.073079 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c8397c9e-0f6c-42d8-9fb9-19214add9e81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11cd7f17ba014170935f1c038cca0eee'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:58.073694 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c8397c9e-0f6c-42d8-9fb9-19214add9e81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11cd7f17ba014170935f1c038cca0eee'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:58.074311 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c8397c9e-0f6c-42d8-9fb9-19214add9e81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11cd7f17ba014170935f1c038cca0eee'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:58.091628 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1273/2527] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:16:57 2022] DELETE /identity/v3/users/11cd7f17ba014170935f1c038cca0eee => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:58.096648 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52a325-f004-4959-adce-fd4a1baefe43 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:58.097125 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52a325-f004-4959-adce-fd4a1baefe43 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:58.097224 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52a325-f004-4959-adce-fd4a1baefe43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:58.116366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1b52a325-f004-4959-adce-fd4a1baefe43 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:58.117555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1b52a325-f004-4959-adce-fd4a1baefe43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:58.175391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1255/2528] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:58.185063 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8397c9e-0f6c-42d8-9fb9-19214add9e81 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:58.186487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c8397c9e-0f6c-42d8-9fb9-19214add9e81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:58.192298 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:58.245388 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8397c9e-0f6c-42d8-9fb9-19214add9e81 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:58.250025 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568197f-2791-462e-bcf0-375429956897 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:58.250629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568197f-2791-462e-bcf0-375429956897 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:58.250906 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568197f-2791-462e-bcf0-375429956897 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:58.253259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568197f-2791-462e-bcf0-375429956897 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:58.261669 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f568197f-2791-462e-bcf0-375429956897 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:58.263153 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f568197f-2791-462e-bcf0-375429956897 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:58.274830 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1274/2529] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:16:58.705636 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b52a325-f004-4959-adce-fd4a1baefe43 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:58.706522 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1b52a325-f004-4959-adce-fd4a1baefe43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:58.719728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b52a325-f004-4959-adce-fd4a1baefe43 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:58.726215 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13626da-e330-4ba8-b7d4-23f2a9fe80e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:58.726946 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13626da-e330-4ba8-b7d4-23f2a9fe80e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:58.727330 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13626da-e330-4ba8-b7d4-23f2a9fe80e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:58.727811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13626da-e330-4ba8-b7d4-23f2a9fe80e3 admin admin] PATH_INFO: `/v3/projects/7e6487713f214e96afb9a543a9d94430` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:58.729295 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13626da-e330-4ba8-b7d4-23f2a9fe80e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e6487713f214e96afb9a543a9d94430)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:58.731289 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13626da-e330-4ba8-b7d4-23f2a9fe80e3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:58.846317 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d13626da-e330-4ba8-b7d4-23f2a9fe80e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e6487713f214e96afb9a543a9d94430', 'request_id': 'req-d13626da-e330-4ba8-b7d4-23f2a9fe80e3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:58.852562 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d13626da-e330-4ba8-b7d4-23f2a9fe80e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e6487713f214e96afb9a543a9d94430', 'request_id': 'req-d13626da-e330-4ba8-b7d4-23f2a9fe80e3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:59.017366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d13626da-e330-4ba8-b7d4-23f2a9fe80e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e6487713f214e96afb9a543a9d94430', 'request_id': 'req-d13626da-e330-4ba8-b7d4-23f2a9fe80e3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:59.029889 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d13626da-e330-4ba8-b7d4-23f2a9fe80e3 admin admin] The token cache is being invalidate because project 7e6487713f214e96afb9a543a9d94430 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:16:59.030213 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d13626da-e330-4ba8-b7d4-23f2a9fe80e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:59.031000 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d13626da-e330-4ba8-b7d4-23f2a9fe80e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:16:59.032855 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1256/2530] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:16:58 2022] DELETE /identity/v3/projects/7e6487713f214e96afb9a543a9d94430 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:59.037778 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f972c-b83f-4c82-93b5-97560f70e9e0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:59.038266 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f972c-b83f-4c82-93b5-97560f70e9e0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:59.038387 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f972c-b83f-4c82-93b5-97560f70e9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:59.058641 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0d1f972c-b83f-4c82-93b5-97560f70e9e0 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:59.059939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0d1f972c-b83f-4c82-93b5-97560f70e9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:59.128570 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1275/2531] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:59.136661 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea0a4e-60c7-4524-9d5f-1e37c79dd990 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:59.137143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea0a4e-60c7-4524-9d5f-1e37c79dd990 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:59.137279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea0a4e-60c7-4524-9d5f-1e37c79dd990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:59.159402 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9aea0a4e-60c7-4524-9d5f-1e37c79dd990 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:59.160627 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9aea0a4e-60c7-4524-9d5f-1e37c79dd990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:59.168648 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1257/2532] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:59.172895 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f972c-b83f-4c82-93b5-97560f70e9e0 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:59.183753 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f972c-b83f-4c82-93b5-97560f70e9e0 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:59.189523 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab7828-ed77-4d93-a25b-4af64abecf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:59.190131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab7828-ed77-4d93-a25b-4af64abecf3d admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:59.190410 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab7828-ed77-4d93-a25b-4af64abecf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:59.190682 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab7828-ed77-4d93-a25b-4af64abecf3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:59.191354 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab7828-ed77-4d93-a25b-4af64abecf3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:16:59.191660 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab7828-ed77-4d93-a25b-4af64abecf3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:59.193244 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab7828-ed77-4d93-a25b-4af64abecf3d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:59.206469 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1276/2533] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 13:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:16:59.210817 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea0a4e-60c7-4524-9d5f-1e37c79dd990 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:59.221267 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea0a4e-60c7-4524-9d5f-1e37c79dd990 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:59.225966 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40800a9a-d6be-4761-b3f6-b2e5d6d2c04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:59.226644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40800a9a-d6be-4761-b3f6-b2e5d6d2c04d admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:59.226988 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40800a9a-d6be-4761-b3f6-b2e5d6d2c04d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:59.227359 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40800a9a-d6be-4761-b3f6-b2e5d6d2c04d admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:59.228046 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40800a9a-d6be-4761-b3f6-b2e5d6d2c04d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:59.230020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40800a9a-d6be-4761-b3f6-b2e5d6d2c04d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:59.321211 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1258/2534] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:16:59 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:16:59.325556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab7828-ed77-4d93-a25b-4af64abecf3d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:59.337263 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab7828-ed77-4d93-a25b-4af64abecf3d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:59.400279 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015b101-d96d-4ca2-80ba-6cd4777532e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:59.400893 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015b101-d96d-4ca2-80ba-6cd4777532e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:59.401195 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015b101-d96d-4ca2-80ba-6cd4777532e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:59.401492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015b101-d96d-4ca2-80ba-6cd4777532e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:59.402113 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015b101-d96d-4ca2-80ba-6cd4777532e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:59.404064 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015b101-d96d-4ca2-80ba-6cd4777532e2 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:59.486226 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1277/2535] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:16:59 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:16:59.490659 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40800a9a-d6be-4761-b3f6-b2e5d6d2c04d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:59.496734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:59.502146 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40800a9a-d6be-4761-b3f6-b2e5d6d2c04d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:59.506932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c5a49-3e80-4916-83c0-f0762cc9475d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:59.507645 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c5a49-3e80-4916-83c0-f0762cc9475d admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:59.507925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c5a49-3e80-4916-83c0-f0762cc9475d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:59.508202 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c5a49-3e80-4916-83c0-f0762cc9475d admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:59.508881 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841c5a49-3e80-4916-83c0-f0762cc9475d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:59.510471 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841c5a49-3e80-4916-83c0-f0762cc9475d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:59.520871 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1259/2536] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:16:59.525049 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015b101-d96d-4ca2-80ba-6cd4777532e2 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:59.536328 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015b101-d96d-4ca2-80ba-6cd4777532e2 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:59.541052 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf406e7b-2b9b-489c-a2f3-f044206821b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:59.541657 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf406e7b-2b9b-489c-a2f3-f044206821b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:59.541945 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf406e7b-2b9b-489c-a2f3-f044206821b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:59.542215 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf406e7b-2b9b-489c-a2f3-f044206821b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:59.542883 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf406e7b-2b9b-489c-a2f3-f044206821b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:59.544517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf406e7b-2b9b-489c-a2f3-f044206821b5 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:59.556426 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1278/2537] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:16:59.560803 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c5a49-3e80-4916-83c0-f0762cc9475d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:59.571520 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c5a49-3e80-4916-83c0-f0762cc9475d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:16:59.576374 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e22efe-b808-4727-84fc-24e9061b5a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:16:59.577018 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e22efe-b808-4727-84fc-24e9061b5a06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:59.577330 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e22efe-b808-4727-84fc-24e9061b5a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:59.578206 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e22efe-b808-4727-84fc-24e9061b5a06 admin admin] PATH_INFO: `/v3/projects/0874b1e036b34f9188b4b5b003d59742/users/40c6361443a14e9ebd71855ba11cee10/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:59.597085 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e22efe-b808-4727-84fc-24e9061b5a06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0874b1e036b34f9188b4b5b003d59742, user_id=40c6361443a14e9ebd71855ba11cee10, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:16:59.599804 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e22efe-b808-4727-84fc-24e9061b5a06 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:16:59.737772 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1260/2538] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:16:59 2022] PUT /identity/v3/projects/0874b1e036b34f9188b4b5b003d59742/users/40c6361443a14e9ebd71855ba11cee10/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:16:59.752701 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6740a1-0c59-4105-884b-2b0a8075724f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:16:59.752969 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6740a1-0c59-4105-884b-2b0a8075724f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:16:59.753268 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6740a1-0c59-4105-884b-2b0a8075724f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:16:59.775283 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3d6740a1-0c59-4105-884b-2b0a8075724f None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:16:59.776543 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3d6740a1-0c59-4105-884b-2b0a8075724f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:59.783922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:16:59.849179 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1279/2539] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:16:59.861959 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e22efe-b808-4727-84fc-24e9061b5a06 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:16:59.864029 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-67e22efe-b808-4727-84fc-24e9061b5a06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:16:59.932089 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e22efe-b808-4727-84fc-24e9061b5a06 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:00.093812 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9444ec7-6ec5-4d9d-b4fb-898eb4765312 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:00.093812 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9444ec7-6ec5-4d9d-b4fb-898eb4765312 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:00.093812 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9444ec7-6ec5-4d9d-b4fb-898eb4765312 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:00.093812 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9444ec7-6ec5-4d9d-b4fb-898eb4765312 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:00.094710 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9444ec7-6ec5-4d9d-b4fb-898eb4765312 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:00.094710 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9444ec7-6ec5-4d9d-b4fb-898eb4765312 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:00.094710 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1261/2540] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:16:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:17:07.312320 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247a287-bc58-4185-acf4-a09f56b6adfe None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:07.312908 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247a287-bc58-4185-acf4-a09f56b6adfe None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:07.312908 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247a287-bc58-4185-acf4-a09f56b6adfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:07.344586 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0247a287-bc58-4185-acf4-a09f56b6adfe None None] MFA Rules not processed for user `40c6361443a14e9ebd71855ba11cee10`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:07.345829 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0247a287-bc58-4185-acf4-a09f56b6adfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:07.418810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1280/2541] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:17:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:17:07.427666 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9444ec7-6ec5-4d9d-b4fb-898eb4765312 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:07.439956 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9444ec7-6ec5-4d9d-b4fb-898eb4765312 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:07.445231 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec62f46c-c082-4ea1-9930-8525cb504048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:07.449437 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62f46c-c082-4ea1-9930-8525cb504048 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:07.449437 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62f46c-c082-4ea1-9930-8525cb504048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:07.449437 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62f46c-c082-4ea1-9930-8525cb504048 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:07.449437 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec62f46c-c082-4ea1-9930-8525cb504048 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:07.449437 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec62f46c-c082-4ea1-9930-8525cb504048 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:07.530251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1262/2542] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:17:07 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:17:07.531739 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247a287-bc58-4185-acf4-a09f56b6adfe None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:07.543842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247a287-bc58-4185-acf4-a09f56b6adfe None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:07.608049 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb8286-5106-46e1-942e-8775263fe982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:07.609249 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb8286-5106-46e1-942e-8775263fe982 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:07.609533 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb8286-5106-46e1-942e-8775263fe982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:07.609806 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb8286-5106-46e1-942e-8775263fe982 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:07.610439 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb8286-5106-46e1-942e-8775263fe982 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:07.612735 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb8286-5106-46e1-942e-8775263fe982 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:07.687967 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1281/2543] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:17:07 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:17:07.689222 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec62f46c-c082-4ea1-9930-8525cb504048 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:07.699817 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec62f46c-c082-4ea1-9930-8525cb504048 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:07.705139 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905cb1bf-6fab-476a-a2a3-0841c677731e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:07.705689 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905cb1bf-6fab-476a-a2a3-0841c677731e admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:07.705993 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905cb1bf-6fab-476a-a2a3-0841c677731e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:07.706300 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905cb1bf-6fab-476a-a2a3-0841c677731e admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:07.706969 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905cb1bf-6fab-476a-a2a3-0841c677731e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:07.708704 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905cb1bf-6fab-476a-a2a3-0841c677731e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:07.720506 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1263/2544] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:17:07.724677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb8286-5106-46e1-942e-8775263fe982 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:07.735706 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb8286-5106-46e1-942e-8775263fe982 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:07.740706 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c68cff-5b8a-4ef5-a9da-590ac26e244f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:07.741352 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c68cff-5b8a-4ef5-a9da-590ac26e244f admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:07.741634 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c68cff-5b8a-4ef5-a9da-590ac26e244f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:07.741911 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c68cff-5b8a-4ef5-a9da-590ac26e244f admin admin] PATH_INFO: `/v3/projects/50d85908bf494c6ea3ba23fd6291cb02/users/f9fb27456558481fb89de506304d8606/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:07.757271 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c68cff-5b8a-4ef5-a9da-590ac26e244f admin admin] RBAC: Authorizing `identity:create_grant(project_id=50d85908bf494c6ea3ba23fd6291cb02, user_id=f9fb27456558481fb89de506304d8606, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:07.759967 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c68cff-5b8a-4ef5-a9da-590ac26e244f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:07.908971 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1282/2545] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:17:07 2022] PUT /identity/v3/projects/50d85908bf494c6ea3ba23fd6291cb02/users/f9fb27456558481fb89de506304d8606/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:17:07.913051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905cb1bf-6fab-476a-a2a3-0841c677731e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:07.923876 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905cb1bf-6fab-476a-a2a3-0841c677731e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:07.978649 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a03204-21f4-4959-8f4e-bb6fb3c6cfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:07.979204 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a03204-21f4-4959-8f4e-bb6fb3c6cfae admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:07.979487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a03204-21f4-4959-8f4e-bb6fb3c6cfae admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:07.979763 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a03204-21f4-4959-8f4e-bb6fb3c6cfae admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:07.980395 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a03204-21f4-4959-8f4e-bb6fb3c6cfae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:07.981988 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a03204-21f4-4959-8f4e-bb6fb3c6cfae admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:07.992472 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1264/2546] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:17:07.996755 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c68cff-5b8a-4ef5-a9da-590ac26e244f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:08.006951 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c68cff-5b8a-4ef5-a9da-590ac26e244f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:08.011569 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e292322d-10a0-441a-9eaa-1ab168e35c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:08.012222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292322d-10a0-441a-9eaa-1ab168e35c4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:08.012503 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292322d-10a0-441a-9eaa-1ab168e35c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:08.012781 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292322d-10a0-441a-9eaa-1ab168e35c4f admin admin] PATH_INFO: `/v3/projects/50d85908bf494c6ea3ba23fd6291cb02/users/f9fb27456558481fb89de506304d8606/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:08.014757 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e292322d-10a0-441a-9eaa-1ab168e35c4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=50d85908bf494c6ea3ba23fd6291cb02, user_id=f9fb27456558481fb89de506304d8606, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:08.017447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e292322d-10a0-441a-9eaa-1ab168e35c4f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:08.019678 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:08.028184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1283/2547] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:17:07 2022] PUT /identity/v3/projects/50d85908bf494c6ea3ba23fd6291cb02/users/f9fb27456558481fb89de506304d8606/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:17:10.413479 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a03204-21f4-4959-8f4e-bb6fb3c6cfae admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:10.415341 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-85a03204-21f4-4959-8f4e-bb6fb3c6cfae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:10.494992 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:10.548871 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a03204-21f4-4959-8f4e-bb6fb3c6cfae admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:10.621042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d9cfc-e18c-45a8-aacd-bb0dcc2d90f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:10.621735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d9cfc-e18c-45a8-aacd-bb0dcc2d90f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:10.622144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d9cfc-e18c-45a8-aacd-bb0dcc2d90f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:10.622566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d9cfc-e18c-45a8-aacd-bb0dcc2d90f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:10.625138 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9e8d9cfc-e18c-45a8-aacd-bb0dcc2d90f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:10.740714 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8d9cfc-e18c-45a8-aacd-bb0dcc2d90f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:10.742295 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8d9cfc-e18c-45a8-aacd-bb0dcc2d90f0 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:10.758996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1265/2548] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:17:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:17:14.458218 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd22e2-337c-4b22-be83-51128b1b939d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:14.458684 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd22e2-337c-4b22-be83-51128b1b939d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:14.458774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd22e2-337c-4b22-be83-51128b1b939d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:14.489663 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-8dbd22e2-337c-4b22-be83-51128b1b939d None None] MFA Rules not processed for user `f9fb27456558481fb89de506304d8606`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:14.490897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8dbd22e2-337c-4b22-be83-51128b1b939d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:14.567735 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1284/2549] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:17:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:17:14.621320 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d9cfc-e18c-45a8-aacd-bb0dcc2d90f0 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:14.622754 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9e8d9cfc-e18c-45a8-aacd-bb0dcc2d90f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:14.705599 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d9cfc-e18c-45a8-aacd-bb0dcc2d90f0 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:14.715467 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e863ef-a2b4-4a42-b196-3eaca3294067 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:14.716102 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e863ef-a2b4-4a42-b196-3eaca3294067 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:14.716384 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e863ef-a2b4-4a42-b196-3eaca3294067 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:14.716658 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e863ef-a2b4-4a42-b196-3eaca3294067 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:14.762034 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:14.777702 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e863ef-a2b4-4a42-b196-3eaca3294067 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:14.779335 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e863ef-a2b4-4a42-b196-3eaca3294067 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:14.788495 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1266/2550] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:17:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:17:17.875499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058d1e0-b672-4afa-807a-bd3eba4ea356 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:17.876028 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058d1e0-b672-4afa-807a-bd3eba4ea356 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:17.876846 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058d1e0-b672-4afa-807a-bd3eba4ea356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:17.897004 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e058d1e0-b672-4afa-807a-bd3eba4ea356 None None] MFA Rules not processed for user `40c6361443a14e9ebd71855ba11cee10`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:17.898251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e058d1e0-b672-4afa-807a-bd3eba4ea356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:17.905494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1285/2551] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:17:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:17:21.785319 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e863ef-a2b4-4a42-b196-3eaca3294067 service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:21.786869 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b5e863ef-a2b4-4a42-b196-3eaca3294067 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:21.853498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e863ef-a2b4-4a42-b196-3eaca3294067 service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:21.858510 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f687d800-cb36-4fe4-a715-78346f8096be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:21.859252 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687d800-cb36-4fe4-a715-78346f8096be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:21.859449 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687d800-cb36-4fe4-a715-78346f8096be admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:21.859730 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687d800-cb36-4fe4-a715-78346f8096be admin admin] PATH_INFO: `/v3/users/afd1122093f14d1f92ddaccfac47eb39` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:21.860921 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f687d800-cb36-4fe4-a715-78346f8096be admin admin] RBAC: Authorizing `identity:delete_user(user_id=afd1122093f14d1f92ddaccfac47eb39)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:21.863861 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f687d800-cb36-4fe4-a715-78346f8096be admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:22.113241 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f687d800-cb36-4fe4-a715-78346f8096be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afd1122093f14d1f92ddaccfac47eb39'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:22.182473 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f687d800-cb36-4fe4-a715-78346f8096be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afd1122093f14d1f92ddaccfac47eb39'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:22.183163 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f687d800-cb36-4fe4-a715-78346f8096be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afd1122093f14d1f92ddaccfac47eb39'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:22.183834 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f687d800-cb36-4fe4-a715-78346f8096be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afd1122093f14d1f92ddaccfac47eb39'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:22.193157 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f687d800-cb36-4fe4-a715-78346f8096be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afd1122093f14d1f92ddaccfac47eb39'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:22.213121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1267/2552] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:17:21 2022] DELETE /identity/v3/users/afd1122093f14d1f92ddaccfac47eb39 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:17:22.217831 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14abdfb-7e16-4b02-8d27-de536cf6fa8f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:22.218905 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14abdfb-7e16-4b02-8d27-de536cf6fa8f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:22.218905 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14abdfb-7e16-4b02-8d27-de536cf6fa8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:22.239747 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b14abdfb-7e16-4b02-8d27-de536cf6fa8f None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:22.240986 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b14abdfb-7e16-4b02-8d27-de536cf6fa8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:22.310604 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1286/2553] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:17:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:17:22.320514 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f687d800-cb36-4fe4-a715-78346f8096be admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:22.322107 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f687d800-cb36-4fe4-a715-78346f8096be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:22.385270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f687d800-cb36-4fe4-a715-78346f8096be admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:22.388939 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61d1d4-e1d3-42cd-b9df-19c26d7281de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:22.389523 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61d1d4-e1d3-42cd-b9df-19c26d7281de service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:22.389820 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61d1d4-e1d3-42cd-b9df-19c26d7281de service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:22.390058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61d1d4-e1d3-42cd-b9df-19c26d7281de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:22.402400 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61d1d4-e1d3-42cd-b9df-19c26d7281de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:22.402737 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61d1d4-e1d3-42cd-b9df-19c26d7281de service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:22.413707 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1268/2554] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:17:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:17:22.830596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14abdfb-7e16-4b02-8d27-de536cf6fa8f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:22.835238 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b14abdfb-7e16-4b02-8d27-de536cf6fa8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:22.856419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14abdfb-7e16-4b02-8d27-de536cf6fa8f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:22.863460 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d18bf4-21d3-4934-a262-ec56e38982aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:22.864901 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d18bf4-21d3-4934-a262-ec56e38982aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:22.865537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d18bf4-21d3-4934-a262-ec56e38982aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:22.866046 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d18bf4-21d3-4934-a262-ec56e38982aa admin admin] PATH_INFO: `/v3/projects/cd8a24b99e604a68bb968f47897306d1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:22.867379 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d18bf4-21d3-4934-a262-ec56e38982aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd8a24b99e604a68bb968f47897306d1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:22.870022 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d18bf4-21d3-4934-a262-ec56e38982aa admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:22.975947 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-54d18bf4-21d3-4934-a262-ec56e38982aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd8a24b99e604a68bb968f47897306d1', 'request_id': 'req-54d18bf4-21d3-4934-a262-ec56e38982aa'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:23.002995 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-54d18bf4-21d3-4934-a262-ec56e38982aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd8a24b99e604a68bb968f47897306d1', 'request_id': 'req-54d18bf4-21d3-4934-a262-ec56e38982aa'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:23.079686 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-54d18bf4-21d3-4934-a262-ec56e38982aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd8a24b99e604a68bb968f47897306d1', 'request_id': 'req-54d18bf4-21d3-4934-a262-ec56e38982aa'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:23.083396 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:23.093045 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-54d18bf4-21d3-4934-a262-ec56e38982aa admin admin] The token cache is being invalidate because project cd8a24b99e604a68bb968f47897306d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:17:23.094781 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-54d18bf4-21d3-4934-a262-ec56e38982aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:23.094781 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-54d18bf4-21d3-4934-a262-ec56e38982aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:23.095863 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1287/2555] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:17:22 2022] DELETE /identity/v3/projects/cd8a24b99e604a68bb968f47897306d1 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:17:23.100554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61d1d4-e1d3-42cd-b9df-19c26d7281de service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:23.105844 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:23.112625 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61d1d4-e1d3-42cd-b9df-19c26d7281de service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:23.177112 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f176ca7-ffe4-4fb0-8335-aa81f16b6ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:23.177805 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f176ca7-ffe4-4fb0-8335-aa81f16b6ccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:23.178085 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f176ca7-ffe4-4fb0-8335-aa81f16b6ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:23.178360 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f176ca7-ffe4-4fb0-8335-aa81f16b6ccd admin admin] PATH_INFO: `/v3/users/8a7641e74e3946c599a8bf8aa3125f81` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:23.179536 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f176ca7-ffe4-4fb0-8335-aa81f16b6ccd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a7641e74e3946c599a8bf8aa3125f81)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:23.183267 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f176ca7-ffe4-4fb0-8335-aa81f16b6ccd admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:23.356577 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7f176ca7-ffe4-4fb0-8335-aa81f16b6ccd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a7641e74e3946c599a8bf8aa3125f81'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:23.416543 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7f176ca7-ffe4-4fb0-8335-aa81f16b6ccd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a7641e74e3946c599a8bf8aa3125f81'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:23.417381 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7f176ca7-ffe4-4fb0-8335-aa81f16b6ccd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a7641e74e3946c599a8bf8aa3125f81'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:23.417957 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7f176ca7-ffe4-4fb0-8335-aa81f16b6ccd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a7641e74e3946c599a8bf8aa3125f81'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:23.432669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7f176ca7-ffe4-4fb0-8335-aa81f16b6ccd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a7641e74e3946c599a8bf8aa3125f81'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:23.451552 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1269/2556] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:17:23 2022] DELETE /identity/v3/users/8a7641e74e3946c599a8bf8aa3125f81 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:17:23.875428 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d18bf4-21d3-4934-a262-ec56e38982aa admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:23.876950 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-54d18bf4-21d3-4934-a262-ec56e38982aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:23.891343 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d18bf4-21d3-4934-a262-ec56e38982aa admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:23.957928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb100a-0896-4d44-8fa4-b3080263539e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:23.958573 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb100a-0896-4d44-8fa4-b3080263539e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:23.958850 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb100a-0896-4d44-8fa4-b3080263539e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:23.959118 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb100a-0896-4d44-8fa4-b3080263539e admin admin] PATH_INFO: `/v3/projects/59a729419e70483fbde2ae60ebda5df8` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:23.960295 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb100a-0896-4d44-8fa4-b3080263539e admin admin] RBAC: Authorizing `identity:delete_project(project_id=59a729419e70483fbde2ae60ebda5df8)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:23.962193 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb100a-0896-4d44-8fa4-b3080263539e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:24.070716 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bceb100a-0896-4d44-8fa4-b3080263539e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59a729419e70483fbde2ae60ebda5df8', 'request_id': 'req-bceb100a-0896-4d44-8fa4-b3080263539e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:24.079465 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bceb100a-0896-4d44-8fa4-b3080263539e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59a729419e70483fbde2ae60ebda5df8', 'request_id': 'req-bceb100a-0896-4d44-8fa4-b3080263539e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:24.142165 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bceb100a-0896-4d44-8fa4-b3080263539e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59a729419e70483fbde2ae60ebda5df8', 'request_id': 'req-bceb100a-0896-4d44-8fa4-b3080263539e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:24.154232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bceb100a-0896-4d44-8fa4-b3080263539e admin admin] The token cache is being invalidate because project 59a729419e70483fbde2ae60ebda5df8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:17:24.154524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bceb100a-0896-4d44-8fa4-b3080263539e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:24.155167 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bceb100a-0896-4d44-8fa4-b3080263539e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:24.156663 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1288/2557] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:17:23 2022] DELETE /identity/v3/projects/59a729419e70483fbde2ae60ebda5df8 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:17:24.162404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ee546-bcfc-4b6e-b8e8-1f0ca0331f5c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:24.162855 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ee546-bcfc-4b6e-b8e8-1f0ca0331f5c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:24.162955 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ee546-bcfc-4b6e-b8e8-1f0ca0331f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:24.183937 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-759ee546-bcfc-4b6e-b8e8-1f0ca0331f5c None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:24.185192 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-759ee546-bcfc-4b6e-b8e8-1f0ca0331f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:24.245959 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1270/2558] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:17:24.253992 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe86c14-caf7-4563-be21-9a44e5e2774e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:24.253992 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe86c14-caf7-4563-be21-9a44e5e2774e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:24.253992 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe86c14-caf7-4563-be21-9a44e5e2774e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:24.273731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0fe86c14-caf7-4563-be21-9a44e5e2774e None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:24.274975 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0fe86c14-caf7-4563-be21-9a44e5e2774e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:24.283770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1289/2559] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:17:24.288688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ee546-bcfc-4b6e-b8e8-1f0ca0331f5c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:24.298480 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ee546-bcfc-4b6e-b8e8-1f0ca0331f5c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:24.303767 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f9b3d5-091f-4fbd-b836-b44b1f4990c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:24.304367 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9b3d5-091f-4fbd-b836-b44b1f4990c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:24.304642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9b3d5-091f-4fbd-b836-b44b1f4990c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:24.304910 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9b3d5-091f-4fbd-b836-b44b1f4990c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:24.305551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f9b3d5-091f-4fbd-b836-b44b1f4990c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:17:24.305850 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f9b3d5-091f-4fbd-b836-b44b1f4990c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:24.307492 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f9b3d5-091f-4fbd-b836-b44b1f4990c2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:24.319908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1271/2560] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 13:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:17:24.324160 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe86c14-caf7-4563-be21-9a44e5e2774e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:24.333597 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe86c14-caf7-4563-be21-9a44e5e2774e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:24.338270 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ed3e4-8691-478a-86d2-75a3699b4903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:24.338865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ed3e4-8691-478a-86d2-75a3699b4903 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:24.339148 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ed3e4-8691-478a-86d2-75a3699b4903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:24.339458 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ed3e4-8691-478a-86d2-75a3699b4903 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:24.340084 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ed3e4-8691-478a-86d2-75a3699b4903 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:24.341969 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ed3e4-8691-478a-86d2-75a3699b4903 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:24.424462 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1290/2561] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:17:24 2022] POST /identity/v3/projects => generated 369 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:17:24.428586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f9b3d5-091f-4fbd-b836-b44b1f4990c2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:24.439147 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f9b3d5-091f-4fbd-b836-b44b1f4990c2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:24.444219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:24.501603 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f2989-85ab-4da9-a413-0d21743d396f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:24.505610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f2989-85ab-4da9-a413-0d21743d396f admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:24.505610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f2989-85ab-4da9-a413-0d21743d396f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:24.505610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f2989-85ab-4da9-a413-0d21743d396f admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:24.505610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1f2989-85ab-4da9-a413-0d21743d396f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:24.506311 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1f2989-85ab-4da9-a413-0d21743d396f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:24.577143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1272/2562] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:17:24 2022] POST /identity/v3/users => generated 285 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:17:24.581132 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ed3e4-8691-478a-86d2-75a3699b4903 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:24.592401 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ed3e4-8691-478a-86d2-75a3699b4903 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:24.597781 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f80ed-70bc-41fa-88d2-ab74b7f41f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:24.598371 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f80ed-70bc-41fa-88d2-ab74b7f41f20 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:24.598647 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f80ed-70bc-41fa-88d2-ab74b7f41f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:24.598940 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f80ed-70bc-41fa-88d2-ab74b7f41f20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:24.599678 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7f80ed-70bc-41fa-88d2-ab74b7f41f20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:24.601309 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7f80ed-70bc-41fa-88d2-ab74b7f41f20 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:24.612170 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1291/2563] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:17:24.617113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f2989-85ab-4da9-a413-0d21743d396f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:24.627149 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f2989-85ab-4da9-a413-0d21743d396f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:24.632463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0d27a-f4e6-4f86-bcd1-504a02657fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:24.638895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0d27a-f4e6-4f86-bcd1-504a02657fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:24.638895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0d27a-f4e6-4f86-bcd1-504a02657fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:24.638895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0d27a-f4e6-4f86-bcd1-504a02657fd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:24.638895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0d27a-f4e6-4f86-bcd1-504a02657fd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:24.638895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0d27a-f4e6-4f86-bcd1-504a02657fd2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:24.647545 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1273/2564] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:17:24.652623 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f80ed-70bc-41fa-88d2-ab74b7f41f20 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:24.663728 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f80ed-70bc-41fa-88d2-ab74b7f41f20 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:24.673026 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf236cc-0ce9-40d3-854e-f3844ec58b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:24.674091 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf236cc-0ce9-40d3-854e-f3844ec58b51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:24.674091 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf236cc-0ce9-40d3-854e-f3844ec58b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:24.674350 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf236cc-0ce9-40d3-854e-f3844ec58b51 admin admin] PATH_INFO: `/v3/projects/4e13ab125a864fd39a6abba68a411b1c/users/aeec9d08fc9f4d3d9cd170d71264e26b/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:24.692874 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf236cc-0ce9-40d3-854e-f3844ec58b51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e13ab125a864fd39a6abba68a411b1c, user_id=aeec9d08fc9f4d3d9cd170d71264e26b, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:24.697028 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf236cc-0ce9-40d3-854e-f3844ec58b51 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:24.794360 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1292/2565] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:17:24 2022] PUT /identity/v3/projects/4e13ab125a864fd39a6abba68a411b1c/users/aeec9d08fc9f4d3d9cd170d71264e26b/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:17:24.806798 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c574c0-0bd9-4f7a-8335-a4aef156478e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:24.807166 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c574c0-0bd9-4f7a-8335-a4aef156478e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:24.807474 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c574c0-0bd9-4f7a-8335-a4aef156478e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:24.836237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a2c574c0-0bd9-4f7a-8335-a4aef156478e None None] MFA Rules not processed for user `aeec9d08fc9f4d3d9cd170d71264e26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:24.837463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a2c574c0-0bd9-4f7a-8335-a4aef156478e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:24.903478 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1274/2566] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:17:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:17:24.907961 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf236cc-0ce9-40d3-854e-f3844ec58b51 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:24.934130 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:24.973593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf236cc-0ce9-40d3-854e-f3844ec58b51 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:24.978465 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9db5eb-bb9c-4aa3-912f-f8c9a08f8a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:24.985700 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9db5eb-bb9c-4aa3-912f-f8c9a08f8a13 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:24.985700 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9db5eb-bb9c-4aa3-912f-f8c9a08f8a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:24.985700 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9db5eb-bb9c-4aa3-912f-f8c9a08f8a13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:24.985700 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9db5eb-bb9c-4aa3-912f-f8c9a08f8a13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:24.985700 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9db5eb-bb9c-4aa3-912f-f8c9a08f8a13 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:25.069808 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1293/2567] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:17:24 2022] POST /identity/v3/projects => generated 369 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:17:25.073809 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c574c0-0bd9-4f7a-8335-a4aef156478e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:25.086895 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c574c0-0bd9-4f7a-8335-a4aef156478e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:25.148100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a31304-8ea1-43b5-b108-fd72f4abf9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:25.148720 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a31304-8ea1-43b5-b108-fd72f4abf9ea admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:25.148997 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a31304-8ea1-43b5-b108-fd72f4abf9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:25.149267 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a31304-8ea1-43b5-b108-fd72f4abf9ea admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:25.149886 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a31304-8ea1-43b5-b108-fd72f4abf9ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:25.151845 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a31304-8ea1-43b5-b108-fd72f4abf9ea admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:25.252421 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1275/2568] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:17:25 2022] POST /identity/v3/users => generated 291 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:17:25.256871 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9db5eb-bb9c-4aa3-912f-f8c9a08f8a13 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:25.267666 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9db5eb-bb9c-4aa3-912f-f8c9a08f8a13 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:25.273157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a664ae2f-8364-42ea-a464-614c04e8a780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:25.273753 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664ae2f-8364-42ea-a464-614c04e8a780 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:25.274034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664ae2f-8364-42ea-a464-614c04e8a780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:25.274309 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664ae2f-8364-42ea-a464-614c04e8a780 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:25.274954 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a664ae2f-8364-42ea-a464-614c04e8a780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:25.276617 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a664ae2f-8364-42ea-a464-614c04e8a780 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:25.287737 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1294/2569] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:17:25.291959 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a31304-8ea1-43b5-b108-fd72f4abf9ea admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:25.302664 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a31304-8ea1-43b5-b108-fd72f4abf9ea admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:25.307218 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9e740-87a1-4eaf-b9ff-3fda4699edf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:25.307878 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9e740-87a1-4eaf-b9ff-3fda4699edf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:25.308157 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9e740-87a1-4eaf-b9ff-3fda4699edf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:25.308426 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9e740-87a1-4eaf-b9ff-3fda4699edf2 admin admin] PATH_INFO: `/v3/projects/eb8bb769ae914ffea5113d73596302b4/users/13be2937b5544225b09f1fe2ebfa59dc/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:25.310953 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:25.324837 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9e740-87a1-4eaf-b9ff-3fda4699edf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb8bb769ae914ffea5113d73596302b4, user_id=13be2937b5544225b09f1fe2ebfa59dc, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:25.328054 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9e740-87a1-4eaf-b9ff-3fda4699edf2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:25.417915 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1276/2570] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:17:25 2022] PUT /identity/v3/projects/eb8bb769ae914ffea5113d73596302b4/users/13be2937b5544225b09f1fe2ebfa59dc/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:17:25.421942 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a664ae2f-8364-42ea-a464-614c04e8a780 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:25.432623 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a664ae2f-8364-42ea-a464-614c04e8a780 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:25.491267 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf979785-4aa4-4cd6-b32f-e68f18b06c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:25.491882 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf979785-4aa4-4cd6-b32f-e68f18b06c8a admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:25.492163 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf979785-4aa4-4cd6-b32f-e68f18b06c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:25.492437 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf979785-4aa4-4cd6-b32f-e68f18b06c8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:25.493074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf979785-4aa4-4cd6-b32f-e68f18b06c8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:25.494693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf979785-4aa4-4cd6-b32f-e68f18b06c8a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:25.505499 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1295/2571] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:17:25.509616 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9e740-87a1-4eaf-b9ff-3fda4699edf2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:25.520216 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9e740-87a1-4eaf-b9ff-3fda4699edf2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:25.525457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b210cf6-9f94-4dc2-b8d8-009ddac3de4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:25.526104 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b210cf6-9f94-4dc2-b8d8-009ddac3de4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:25.526384 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b210cf6-9f94-4dc2-b8d8-009ddac3de4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:25.526685 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b210cf6-9f94-4dc2-b8d8-009ddac3de4a admin admin] PATH_INFO: `/v3/projects/eb8bb769ae914ffea5113d73596302b4/users/13be2937b5544225b09f1fe2ebfa59dc/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:25.528686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b210cf6-9f94-4dc2-b8d8-009ddac3de4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb8bb769ae914ffea5113d73596302b4, user_id=13be2937b5544225b09f1fe2ebfa59dc, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:25.531392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b210cf6-9f94-4dc2-b8d8-009ddac3de4a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:25.541546 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1277/2572] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:17:25 2022] PUT /identity/v3/projects/eb8bb769ae914ffea5113d73596302b4/users/13be2937b5544225b09f1fe2ebfa59dc/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:17:25.552164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93bcc0e-5753-4d40-97f4-6235e6d96ef7 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:25.552455 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93bcc0e-5753-4d40-97f4-6235e6d96ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:25.552746 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93bcc0e-5753-4d40-97f4-6235e6d96ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:25.580821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d93bcc0e-5753-4d40-97f4-6235e6d96ef7 None None] MFA Rules not processed for user `13be2937b5544225b09f1fe2ebfa59dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:25.582265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d93bcc0e-5753-4d40-97f4-6235e6d96ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:25.651019 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1296/2573] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:17:25 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:17:25.663098 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b210cf6-9f94-4dc2-b8d8-009ddac3de4a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:25.664796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3b210cf6-9f94-4dc2-b8d8-009ddac3de4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:25.724736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b210cf6-9f94-4dc2-b8d8-009ddac3de4a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:25.728438 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619b8c4-5fa1-4505-8cbb-5888874f9d30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:25.729071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619b8c4-5fa1-4505-8cbb-5888874f9d30 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:25.729347 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619b8c4-5fa1-4505-8cbb-5888874f9d30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:25.729615 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619b8c4-5fa1-4505-8cbb-5888874f9d30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:25.740137 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5619b8c4-5fa1-4505-8cbb-5888874f9d30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:25.741587 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5619b8c4-5fa1-4505-8cbb-5888874f9d30 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:25.755732 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1278/2574] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:17:25 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:17:25.860406 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93bcc0e-5753-4d40-97f4-6235e6d96ef7 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:25.861849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d93bcc0e-5753-4d40-97f4-6235e6d96ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:25.875523 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93bcc0e-5753-4d40-97f4-6235e6d96ef7 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:25.879646 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bcf23b-ac52-4652-8c55-8713e3aa1659 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:25.880250 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcf23b-ac52-4652-8c55-8713e3aa1659 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:25.880527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcf23b-ac52-4652-8c55-8713e3aa1659 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:25.880799 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcf23b-ac52-4652-8c55-8713e3aa1659 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:25.896604 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:25.941348 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bcf23b-ac52-4652-8c55-8713e3aa1659 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:25.943826 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bcf23b-ac52-4652-8c55-8713e3aa1659 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:25.953403 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1297/2575] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:17:25 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:17:30.579775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619b8c4-5fa1-4505-8cbb-5888874f9d30 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:30.641864 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619b8c4-5fa1-4505-8cbb-5888874f9d30 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:30.646298 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047cce2-b04c-49e6-b802-6a0d15104ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:30.646918 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047cce2-b04c-49e6-b802-6a0d15104ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:30.647228 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047cce2-b04c-49e6-b802-6a0d15104ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:30.648115 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047cce2-b04c-49e6-b802-6a0d15104ea4 admin admin] PATH_INFO: `/v3/users/aeec9d08fc9f4d3d9cd170d71264e26b` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:30.649328 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5047cce2-b04c-49e6-b802-6a0d15104ea4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeec9d08fc9f4d3d9cd170d71264e26b)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:30.651275 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5047cce2-b04c-49e6-b802-6a0d15104ea4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:30.806617 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5047cce2-b04c-49e6-b802-6a0d15104ea4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeec9d08fc9f4d3d9cd170d71264e26b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:30.850813 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5047cce2-b04c-49e6-b802-6a0d15104ea4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeec9d08fc9f4d3d9cd170d71264e26b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:30.851468 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5047cce2-b04c-49e6-b802-6a0d15104ea4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeec9d08fc9f4d3d9cd170d71264e26b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:30.852088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5047cce2-b04c-49e6-b802-6a0d15104ea4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeec9d08fc9f4d3d9cd170d71264e26b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:30.859165 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5047cce2-b04c-49e6-b802-6a0d15104ea4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeec9d08fc9f4d3d9cd170d71264e26b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:30.867321 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:30.877039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1279/2576] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:17:30 2022] DELETE /identity/v3/users/aeec9d08fc9f4d3d9cd170d71264e26b => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:17:30.881505 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1473e98-d3b9-4356-a735-1c8fe1aefc64 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:30.882632 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1473e98-d3b9-4356-a735-1c8fe1aefc64 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:30.882889 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1473e98-d3b9-4356-a735-1c8fe1aefc64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:30.903269 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c1473e98-d3b9-4356-a735-1c8fe1aefc64 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:30.904377 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c1473e98-d3b9-4356-a735-1c8fe1aefc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:30.962927 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1298/2577] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:17:30.972865 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047cce2-b04c-49e6-b802-6a0d15104ea4 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:30.974453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5047cce2-b04c-49e6-b802-6a0d15104ea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:31.033809 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047cce2-b04c-49e6-b802-6a0d15104ea4 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:31.037543 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253542ba-2aac-4d09-86ac-363a5579601c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:31.038393 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253542ba-2aac-4d09-86ac-363a5579601c service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:31.038736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253542ba-2aac-4d09-86ac-363a5579601c service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:31.039011 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253542ba-2aac-4d09-86ac-363a5579601c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:31.048139 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253542ba-2aac-4d09-86ac-363a5579601c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:31.049539 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253542ba-2aac-4d09-86ac-363a5579601c service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:31.064658 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1280/2578] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:17:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:17:31.481933 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1473e98-d3b9-4356-a735-1c8fe1aefc64 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:31.483478 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c1473e98-d3b9-4356-a735-1c8fe1aefc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:31.505379 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1473e98-d3b9-4356-a735-1c8fe1aefc64 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:31.512225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf6184-3f70-4780-917e-ab1008bcc446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:31.513179 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf6184-3f70-4780-917e-ab1008bcc446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:31.522987 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf6184-3f70-4780-917e-ab1008bcc446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:31.523373 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf6184-3f70-4780-917e-ab1008bcc446 admin admin] PATH_INFO: `/v3/projects/4e13ab125a864fd39a6abba68a411b1c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:31.524635 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdf6184-3f70-4780-917e-ab1008bcc446 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e13ab125a864fd39a6abba68a411b1c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:31.526673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdf6184-3f70-4780-917e-ab1008bcc446 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:31.612399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0bdf6184-3f70-4780-917e-ab1008bcc446 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e13ab125a864fd39a6abba68a411b1c', 'request_id': 'req-0bdf6184-3f70-4780-917e-ab1008bcc446'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:31.618971 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0bdf6184-3f70-4780-917e-ab1008bcc446 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e13ab125a864fd39a6abba68a411b1c', 'request_id': 'req-0bdf6184-3f70-4780-917e-ab1008bcc446'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:31.696442 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0bdf6184-3f70-4780-917e-ab1008bcc446 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e13ab125a864fd39a6abba68a411b1c', 'request_id': 'req-0bdf6184-3f70-4780-917e-ab1008bcc446'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:31.711205 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0bdf6184-3f70-4780-917e-ab1008bcc446 admin admin] The token cache is being invalidate because project 4e13ab125a864fd39a6abba68a411b1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:17:31.711515 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0bdf6184-3f70-4780-917e-ab1008bcc446 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:31.712222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0bdf6184-3f70-4780-917e-ab1008bcc446 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:31.713885 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1299/2579] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:17:31 2022] DELETE /identity/v3/projects/4e13ab125a864fd39a6abba68a411b1c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:17:31.718004 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253542ba-2aac-4d09-86ac-363a5579601c service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:31.728868 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253542ba-2aac-4d09-86ac-363a5579601c service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:31.771342 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf6184-3f70-4780-917e-ab1008bcc446 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:31.773060 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0bdf6184-3f70-4780-917e-ab1008bcc446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:31.804748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d919e-9c02-42e3-b991-8eb5efde44b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:31.805390 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d919e-9c02-42e3-b991-8eb5efde44b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:31.805882 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d919e-9c02-42e3-b991-8eb5efde44b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:31.806073 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d919e-9c02-42e3-b991-8eb5efde44b4 admin admin] PATH_INFO: `/v3/users/13be2937b5544225b09f1fe2ebfa59dc` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:31.807464 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d919e-9c02-42e3-b991-8eb5efde44b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13be2937b5544225b09f1fe2ebfa59dc)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:31.809177 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d919e-9c02-42e3-b991-8eb5efde44b4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:31.831810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:31.846613 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf6184-3f70-4780-917e-ab1008bcc446 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:31.850309 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987dc89-0db1-45b2-984f-e476d92d4385 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:31.851479 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987dc89-0db1-45b2-984f-e476d92d4385 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:31.851936 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987dc89-0db1-45b2-984f-e476d92d4385 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:31.852231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987dc89-0db1-45b2-984f-e476d92d4385 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:31.853997 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f987dc89-0db1-45b2-984f-e476d92d4385 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:31.920182 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f987dc89-0db1-45b2-984f-e476d92d4385 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:31.921618 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f987dc89-0db1-45b2-984f-e476d92d4385 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:31.932093 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1300/2580] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:17:31 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:17:31.998427 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6a3d919e-9c02-42e3-b991-8eb5efde44b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13be2937b5544225b09f1fe2ebfa59dc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:32.050529 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6a3d919e-9c02-42e3-b991-8eb5efde44b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13be2937b5544225b09f1fe2ebfa59dc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:32.051398 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6a3d919e-9c02-42e3-b991-8eb5efde44b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13be2937b5544225b09f1fe2ebfa59dc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:32.052104 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6a3d919e-9c02-42e3-b991-8eb5efde44b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13be2937b5544225b09f1fe2ebfa59dc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:32.061784 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6a3d919e-9c02-42e3-b991-8eb5efde44b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13be2937b5544225b09f1fe2ebfa59dc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:32.090941 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1281/2581] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:17:31 2022] DELETE /identity/v3/users/13be2937b5544225b09f1fe2ebfa59dc => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:17:32.619118 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987dc89-0db1-45b2-984f-e476d92d4385 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:32.620768 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f987dc89-0db1-45b2-984f-e476d92d4385 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:32.635296 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987dc89-0db1-45b2-984f-e476d92d4385 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:32.705920 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8373f-bbb9-49d3-9697-fd32b4c6f509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:32.706558 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8373f-bbb9-49d3-9697-fd32b4c6f509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:32.706841 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8373f-bbb9-49d3-9697-fd32b4c6f509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:32.707112 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8373f-bbb9-49d3-9697-fd32b4c6f509 admin admin] PATH_INFO: `/v3/projects/eb8bb769ae914ffea5113d73596302b4` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:32.711995 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8373f-bbb9-49d3-9697-fd32b4c6f509 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb8bb769ae914ffea5113d73596302b4)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:32.711995 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8373f-bbb9-49d3-9697-fd32b4c6f509 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:32.807973 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c2d8373f-bbb9-49d3-9697-fd32b4c6f509 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb8bb769ae914ffea5113d73596302b4', 'request_id': 'req-c2d8373f-bbb9-49d3-9697-fd32b4c6f509'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:32.813804 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c2d8373f-bbb9-49d3-9697-fd32b4c6f509 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb8bb769ae914ffea5113d73596302b4', 'request_id': 'req-c2d8373f-bbb9-49d3-9697-fd32b4c6f509'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:32.919671 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c2d8373f-bbb9-49d3-9697-fd32b4c6f509 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb8bb769ae914ffea5113d73596302b4', 'request_id': 'req-c2d8373f-bbb9-49d3-9697-fd32b4c6f509'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:32.931200 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c2d8373f-bbb9-49d3-9697-fd32b4c6f509 admin admin] The token cache is being invalidate because project eb8bb769ae914ffea5113d73596302b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:17:32.931788 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c2d8373f-bbb9-49d3-9697-fd32b4c6f509 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:32.932202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c2d8373f-bbb9-49d3-9697-fd32b4c6f509 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:32.933683 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1301/2582] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:17:32 2022] DELETE /identity/v3/projects/eb8bb769ae914ffea5113d73596302b4 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:17:32.938534 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f815a-dbb4-4759-a08c-3c32858c6774 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:32.938996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f815a-dbb4-4759-a08c-3c32858c6774 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:32.939086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f815a-dbb4-4759-a08c-3c32858c6774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:32.959511 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-be5f815a-dbb4-4759-a08c-3c32858c6774 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:32.961263 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-be5f815a-dbb4-4759-a08c-3c32858c6774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:32.970216 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:33.022213 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1282/2583] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:17:33.030821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb07df-6478-4465-aec4-63e60d6e3199 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:33.035101 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb07df-6478-4465-aec4-63e60d6e3199 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:33.035101 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb07df-6478-4465-aec4-63e60d6e3199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:33.052092 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-dbcb07df-6478-4465-aec4-63e60d6e3199 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:33.058058 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-dbcb07df-6478-4465-aec4-63e60d6e3199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:33.061360 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1302/2584] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:17:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:17:33.065714 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f815a-dbb4-4759-a08c-3c32858c6774 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:33.076510 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f815a-dbb4-4759-a08c-3c32858c6774 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:33.081110 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3a04c-acd5-490c-8450-5804da27b2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:33.081722 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3a04c-acd5-490c-8450-5804da27b2b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:33.081996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3a04c-acd5-490c-8450-5804da27b2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:33.082271 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3a04c-acd5-490c-8450-5804da27b2b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:33.082914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e3a04c-acd5-490c-8450-5804da27b2b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:17:33.083250 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e3a04c-acd5-490c-8450-5804da27b2b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:33.084886 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e3a04c-acd5-490c-8450-5804da27b2b5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:33.102587 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1283/2585] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 13:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:17:33.103874 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb07df-6478-4465-aec4-63e60d6e3199 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:33.114270 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb07df-6478-4465-aec4-63e60d6e3199 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:33.119490 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5c14f-36ef-4d41-b40c-4a52cd4f77df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:33.121249 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5c14f-36ef-4d41-b40c-4a52cd4f77df admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:33.121249 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5c14f-36ef-4d41-b40c-4a52cd4f77df admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:33.121249 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5c14f-36ef-4d41-b40c-4a52cd4f77df admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:33.121623 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df5c14f-36ef-4d41-b40c-4a52cd4f77df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:33.123195 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df5c14f-36ef-4d41-b40c-4a52cd4f77df admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:33.135398 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:33.186350 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1303/2586] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:17:33 2022] POST /identity/v3/projects => generated 371 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:17:33.190467 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3a04c-acd5-490c-8450-5804da27b2b5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:33.201674 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3a04c-acd5-490c-8450-5804da27b2b5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:33.263326 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1062dd-87c8-44e6-aa5f-b53b9602a2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:33.264246 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1062dd-87c8-44e6-aa5f-b53b9602a2c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:33.264524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1062dd-87c8-44e6-aa5f-b53b9602a2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:33.264796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1062dd-87c8-44e6-aa5f-b53b9602a2c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:33.265431 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1062dd-87c8-44e6-aa5f-b53b9602a2c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:33.267427 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1062dd-87c8-44e6-aa5f-b53b9602a2c8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:33.368775 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1284/2587] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:17:33 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:17:33.370059 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5c14f-36ef-4d41-b40c-4a52cd4f77df admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:33.383210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5c14f-36ef-4d41-b40c-4a52cd4f77df admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:33.390716 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27585021-3cf3-4e62-897a-caef0e11bd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:33.395218 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27585021-3cf3-4e62-897a-caef0e11bd6a admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:33.395218 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27585021-3cf3-4e62-897a-caef0e11bd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:33.395218 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27585021-3cf3-4e62-897a-caef0e11bd6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:33.395218 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27585021-3cf3-4e62-897a-caef0e11bd6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:33.395218 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27585021-3cf3-4e62-897a-caef0e11bd6a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:33.408967 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1304/2588] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:17:33.410212 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1062dd-87c8-44e6-aa5f-b53b9602a2c8 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:33.424026 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1062dd-87c8-44e6-aa5f-b53b9602a2c8 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:33.444193 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e54d7-7e33-4f49-a2ae-ef87223510fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:33.445663 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e54d7-7e33-4f49-a2ae-ef87223510fd admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:33.446331 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e54d7-7e33-4f49-a2ae-ef87223510fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:33.446444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e54d7-7e33-4f49-a2ae-ef87223510fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:33.447054 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e54d7-7e33-4f49-a2ae-ef87223510fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:33.448748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e54d7-7e33-4f49-a2ae-ef87223510fd admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:33.461242 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1285/2589] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:17:33.465270 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27585021-3cf3-4e62-897a-caef0e11bd6a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:33.477056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27585021-3cf3-4e62-897a-caef0e11bd6a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:33.482246 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d832c35-978f-40c9-bcdd-12ff9ae9e151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:33.482907 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d832c35-978f-40c9-bcdd-12ff9ae9e151 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:33.483220 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d832c35-978f-40c9-bcdd-12ff9ae9e151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:33.483500 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d832c35-978f-40c9-bcdd-12ff9ae9e151 admin admin] PATH_INFO: `/v3/projects/8f23c11cbc914ab99f2ca12fcca3efc6/users/c1bb19d59b2047f593f82fd6ca5d5e45/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:33.501748 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d832c35-978f-40c9-bcdd-12ff9ae9e151 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f23c11cbc914ab99f2ca12fcca3efc6, user_id=c1bb19d59b2047f593f82fd6ca5d5e45, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:33.504650 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d832c35-978f-40c9-bcdd-12ff9ae9e151 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:33.571261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1305/2590] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:17:33 2022] PUT /identity/v3/projects/8f23c11cbc914ab99f2ca12fcca3efc6/users/c1bb19d59b2047f593f82fd6ca5d5e45/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:17:33.581057 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5caf4-d4b9-40ce-8350-07d1d8d99b7e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:33.583550 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5caf4-d4b9-40ce-8350-07d1d8d99b7e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:33.584035 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5caf4-d4b9-40ce-8350-07d1d8d99b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:33.620724 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-53d5caf4-d4b9-40ce-8350-07d1d8d99b7e None None] MFA Rules not processed for user `c1bb19d59b2047f593f82fd6ca5d5e45`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:33.621973 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-53d5caf4-d4b9-40ce-8350-07d1d8d99b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:33.692819 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1286/2591] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:17:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:17:33.705573 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d832c35-978f-40c9-bcdd-12ff9ae9e151 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:33.707966 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9d832c35-978f-40c9-bcdd-12ff9ae9e151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:33.774528 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d832c35-978f-40c9-bcdd-12ff9ae9e151 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:33.779132 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d655473-c8cc-407d-b190-f86edf3a5b5b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35dd27fd356241b885706375d880222f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:33.780325 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d655473-c8cc-407d-b190-f86edf3a5b5b service glance] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:33.780325 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d655473-c8cc-407d-b190-f86edf3a5b5b service glance] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:33.787884 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d655473-c8cc-407d-b190-f86edf3a5b5b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:33.790259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d655473-c8cc-407d-b190-f86edf3a5b5b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:33.791858 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d655473-c8cc-407d-b190-f86edf3a5b5b service glance] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:33.800969 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1306/2592] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:17:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:17:43.689659 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=93,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:43.796955 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=44,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:54.230449 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5caf4-d4b9-40ce-8350-07d1d8d99b7e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:54.231896 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-53d5caf4-d4b9-40ce-8350-07d1d8d99b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:54.291278 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5caf4-d4b9-40ce-8350-07d1d8d99b7e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:54.295418 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209f15e-c540-4024-9b2a-7ce068582c22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:54.296022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209f15e-c540-4024-9b2a-7ce068582c22 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:54.296300 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209f15e-c540-4024-9b2a-7ce068582c22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:54.296572 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209f15e-c540-4024-9b2a-7ce068582c22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:54.298296 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1209f15e-c540-4024-9b2a-7ce068582c22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:54.359954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209f15e-c540-4024-9b2a-7ce068582c22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:54.361380 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209f15e-c540-4024-9b2a-7ce068582c22 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:54.371826 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1287/2593] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:17:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:17:58.584110 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d655473-c8cc-407d-b190-f86edf3a5b5b service glance] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:58.586037 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9d655473-c8cc-407d-b190-f86edf3a5b5b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:58.602751 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d655473-c8cc-407d-b190-f86edf3a5b5b service glance] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:58.608558 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f487b0a-223f-4152-9497-0c092b333518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:58.609172 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f487b0a-223f-4152-9497-0c092b333518 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:58.609498 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f487b0a-223f-4152-9497-0c092b333518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:58.609751 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f487b0a-223f-4152-9497-0c092b333518 admin admin] PATH_INFO: `/v3/users/570c2a77b862411083b9444f8292192b` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:58.624629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f487b0a-223f-4152-9497-0c092b333518 admin admin] RBAC: Authorizing `identity:delete_user(user_id=570c2a77b862411083b9444f8292192b)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:58.626580 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f487b0a-223f-4152-9497-0c092b333518 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:58.798892 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-7f487b0a-223f-4152-9497-0c092b333518 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '570c2a77b862411083b9444f8292192b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:58.806585 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-7f487b0a-223f-4152-9497-0c092b333518 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '570c2a77b862411083b9444f8292192b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:58.868087 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-7f487b0a-223f-4152-9497-0c092b333518 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '570c2a77b862411083b9444f8292192b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:58.868643 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-7f487b0a-223f-4152-9497-0c092b333518 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '570c2a77b862411083b9444f8292192b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:58.869265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-7f487b0a-223f-4152-9497-0c092b333518 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '570c2a77b862411083b9444f8292192b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:58.888001 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1307/2594] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:17:58 2022] DELETE /identity/v3/users/570c2a77b862411083b9444f8292192b => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:17:59.127786 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209f15e-c540-4024-9b2a-7ce068582c22 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:59.133860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209f15e-c540-4024-9b2a-7ce068582c22 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:59.195945 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5617e1-b2df-4405-82a3-4fe30663939f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:59.197024 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5617e1-b2df-4405-82a3-4fe30663939f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:59.197542 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5617e1-b2df-4405-82a3-4fe30663939f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:59.198662 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5617e1-b2df-4405-82a3-4fe30663939f admin admin] PATH_INFO: `/v3/projects/a2857015013f466d95f3a9037925c6fc` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:59.200328 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5617e1-b2df-4405-82a3-4fe30663939f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2857015013f466d95f3a9037925c6fc)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:59.202272 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5617e1-b2df-4405-82a3-4fe30663939f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:59.299590 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0b5617e1-b2df-4405-82a3-4fe30663939f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2857015013f466d95f3a9037925c6fc', 'request_id': 'req-0b5617e1-b2df-4405-82a3-4fe30663939f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:59.301509 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:59.305374 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0b5617e1-b2df-4405-82a3-4fe30663939f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2857015013f466d95f3a9037925c6fc', 'request_id': 'req-0b5617e1-b2df-4405-82a3-4fe30663939f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:59.388824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0b5617e1-b2df-4405-82a3-4fe30663939f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2857015013f466d95f3a9037925c6fc', 'request_id': 'req-0b5617e1-b2df-4405-82a3-4fe30663939f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:59.400531 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0b5617e1-b2df-4405-82a3-4fe30663939f admin admin] The token cache is being invalidate because project a2857015013f466d95f3a9037925c6fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:17:59.400823 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0b5617e1-b2df-4405-82a3-4fe30663939f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:59.401525 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0b5617e1-b2df-4405-82a3-4fe30663939f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:17:59.403223 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1288/2595] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:17:59 2022] DELETE /identity/v3/projects/a2857015013f466d95f3a9037925c6fc => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:17:59.408234 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38b548-a2c5-43fe-a3f6-af9b9bc8e9fc None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:59.408604 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38b548-a2c5-43fe-a3f6-af9b9bc8e9fc None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:59.408907 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38b548-a2c5-43fe-a3f6-af9b9bc8e9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:59.429746 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-4b38b548-a2c5-43fe-a3f6-af9b9bc8e9fc None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:59.431060 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4b38b548-a2c5-43fe-a3f6-af9b9bc8e9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:59.497574 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1308/2596] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:17:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:17:59.511093 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e58e7d-6a46-4ea5-9798-606ab24cfff3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:59.517594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e58e7d-6a46-4ea5-9798-606ab24cfff3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:59.517594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e58e7d-6a46-4ea5-9798-606ab24cfff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:59.538589 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-48e58e7d-6a46-4ea5-9798-606ab24cfff3 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:17:59.540631 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-48e58e7d-6a46-4ea5-9798-606ab24cfff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:59.554100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1289/2597] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:17:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:17:59.558640 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b38b548-a2c5-43fe-a3f6-af9b9bc8e9fc None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:59.569620 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b38b548-a2c5-43fe-a3f6-af9b9bc8e9fc None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:59.575363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03fe3e-55d4-44b2-81e7-de6116658bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:59.575649 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03fe3e-55d4-44b2-81e7-de6116658bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:59.575887 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03fe3e-55d4-44b2-81e7-de6116658bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:59.576156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03fe3e-55d4-44b2-81e7-de6116658bd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:59.576803 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc03fe3e-55d4-44b2-81e7-de6116658bd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:17:59.577844 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc03fe3e-55d4-44b2-81e7-de6116658bd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:59.578724 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc03fe3e-55d4-44b2-81e7-de6116658bd2 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:59.592239 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1309/2598] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 13:17:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:17:59.596535 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e58e7d-6a46-4ea5-9798-606ab24cfff3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:59.606365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e58e7d-6a46-4ea5-9798-606ab24cfff3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:59.611349 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a8b32-329c-40f3-932e-5460a7635819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:59.611954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a8b32-329c-40f3-932e-5460a7635819 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:59.613112 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a8b32-329c-40f3-932e-5460a7635819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:59.613384 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a8b32-329c-40f3-932e-5460a7635819 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:59.614012 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a8b32-329c-40f3-932e-5460a7635819 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:59.615993 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a8b32-329c-40f3-932e-5460a7635819 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:59.745489 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1290/2599] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:17:59 2022] POST /identity/v3/projects => generated 379 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:17:59.753140 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03fe3e-55d4-44b2-81e7-de6116658bd2 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:59.764263 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03fe3e-55d4-44b2-81e7-de6116658bd2 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:59.811075 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:17:59.825557 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9dc3b-f9f6-4b4e-83c7-70a6cc5fb3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:59.826167 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9dc3b-f9f6-4b4e-83c7-70a6cc5fb3a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:59.826448 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9dc3b-f9f6-4b4e-83c7-70a6cc5fb3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:59.826903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9dc3b-f9f6-4b4e-83c7-70a6cc5fb3a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:59.827574 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c9dc3b-f9f6-4b4e-83c7-70a6cc5fb3a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:59.829481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c9dc3b-f9f6-4b4e-83c7-70a6cc5fb3a8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:59.872283 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a8b32-329c-40f3-932e-5460a7635819 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:59.874025 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f95a8b32-329c-40f3-932e-5460a7635819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:59.914108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1310/2600] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:17:59 2022] POST /identity/v3/users => generated 290 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:17:59.917760 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9dc3b-f9f6-4b4e-83c7-70a6cc5fb3a8 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:59.928073 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9dc3b-f9f6-4b4e-83c7-70a6cc5fb3a8 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:59.933225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52434dd3-fffc-4f29-b065-7a2e22760edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:59.933815 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52434dd3-fffc-4f29-b065-7a2e22760edb admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:59.934096 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52434dd3-fffc-4f29-b065-7a2e22760edb admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:59.934371 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52434dd3-fffc-4f29-b065-7a2e22760edb admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:59.935008 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52434dd3-fffc-4f29-b065-7a2e22760edb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:59.936731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52434dd3-fffc-4f29-b065-7a2e22760edb admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:59.946483 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a8b32-329c-40f3-932e-5460a7635819 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:59.947289 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1311/2601] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:17:59.951450 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52434dd3-fffc-4f29-b065-7a2e22760edb admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:59.953751 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937dabf-5cd2-496b-a246-7a46230ff7f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:59.954365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937dabf-5cd2-496b-a246-7a46230ff7f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:59.954682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937dabf-5cd2-496b-a246-7a46230ff7f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:59.955003 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937dabf-5cd2-496b-a246-7a46230ff7f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:59.956812 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d937dabf-5cd2-496b-a246-7a46230ff7f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:17:59.961176 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52434dd3-fffc-4f29-b065-7a2e22760edb admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:17:59.965950 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbfa10-13cd-4a5c-88ab-a31caf21dfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:17:59.966527 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbfa10-13cd-4a5c-88ab-a31caf21dfc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:17:59.966803 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbfa10-13cd-4a5c-88ab-a31caf21dfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:17:59.967081 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbfa10-13cd-4a5c-88ab-a31caf21dfc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:17:59.967746 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcbfa10-13cd-4a5c-88ab-a31caf21dfc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:17:59.969396 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcbfa10-13cd-4a5c-88ab-a31caf21dfc8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:17:59.980645 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1312/2602] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:17:59.984628 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbfa10-13cd-4a5c-88ab-a31caf21dfc8 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:17:59.995721 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbfa10-13cd-4a5c-88ab-a31caf21dfc8 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:00.000368 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d1ff1-1aee-49b2-9046-a51beec79c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:00.001004 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d1ff1-1aee-49b2-9046-a51beec79c61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:00.001289 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d1ff1-1aee-49b2-9046-a51beec79c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:00.001567 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d1ff1-1aee-49b2-9046-a51beec79c61 admin admin] PATH_INFO: `/v3/projects/fdf1111675a94fdc9685b4447bb2cb66/users/51bbcebbad284b2fb846eeb3e186ed2b/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:00.015831 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d1ff1-1aee-49b2-9046-a51beec79c61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdf1111675a94fdc9685b4447bb2cb66, user_id=51bbcebbad284b2fb846eeb3e186ed2b, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:00.018480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d1ff1-1aee-49b2-9046-a51beec79c61 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:00.023056 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d937dabf-5cd2-496b-a246-7a46230ff7f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:00.024547 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d937dabf-5cd2-496b-a246-7a46230ff7f5 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:00.034384 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1291/2603] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:17:59 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:18:00.108552 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1313/2604] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:17:59 2022] PUT /identity/v3/projects/fdf1111675a94fdc9685b4447bb2cb66/users/51bbcebbad284b2fb846eeb3e186ed2b/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:18:00.121528 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1280b-2adf-4606-b2ea-0d7e9a05208f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:00.121817 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1280b-2adf-4606-b2ea-0d7e9a05208f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:00.122097 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1280b-2adf-4606-b2ea-0d7e9a05208f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:00.150366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a7f1280b-2adf-4606-b2ea-0d7e9a05208f None None] MFA Rules not processed for user `51bbcebbad284b2fb846eeb3e186ed2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:00.152498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a7f1280b-2adf-4606-b2ea-0d7e9a05208f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:00.197040 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:00.222194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1292/2605] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:18:00.226679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d1ff1-1aee-49b2-9046-a51beec79c61 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:00.299149 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d1ff1-1aee-49b2-9046-a51beec79c61 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:00.305522 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5f9f9-a929-432a-9ac8-0736c4ea5278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:00.305522 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5f9f9-a929-432a-9ac8-0736c4ea5278 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:00.305522 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5f9f9-a929-432a-9ac8-0736c4ea5278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:00.305522 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5f9f9-a929-432a-9ac8-0736c4ea5278 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:00.305919 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5f9f9-a929-432a-9ac8-0736c4ea5278 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:00.308416 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5f9f9-a929-432a-9ac8-0736c4ea5278 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:00.424910 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1314/2606] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:18:00 2022] POST /identity/v3/projects => generated 379 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:18:00.429053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1280b-2adf-4606-b2ea-0d7e9a05208f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:00.440249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1280b-2adf-4606-b2ea-0d7e9a05208f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:00.500425 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc62af8-c41f-4627-88ad-bcc0cb27d432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:00.501037 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc62af8-c41f-4627-88ad-bcc0cb27d432 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:00.501320 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc62af8-c41f-4627-88ad-bcc0cb27d432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:00.501601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc62af8-c41f-4627-88ad-bcc0cb27d432 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:00.502245 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc62af8-c41f-4627-88ad-bcc0cb27d432 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:00.504209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc62af8-c41f-4627-88ad-bcc0cb27d432 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:00.614581 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1293/2607] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:18:00 2022] POST /identity/v3/users => generated 296 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:18:00.618905 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5f9f9-a929-432a-9ac8-0736c4ea5278 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:00.636465 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5f9f9-a929-432a-9ac8-0736c4ea5278 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:00.643560 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4cab7-48e8-4a4d-a9d3-6ecabff6f658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:00.647136 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4cab7-48e8-4a4d-a9d3-6ecabff6f658 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:00.647136 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4cab7-48e8-4a4d-a9d3-6ecabff6f658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:00.647136 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4cab7-48e8-4a4d-a9d3-6ecabff6f658 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:00.647136 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4cab7-48e8-4a4d-a9d3-6ecabff6f658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:00.647631 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4cab7-48e8-4a4d-a9d3-6ecabff6f658 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:00.662361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1315/2608] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:18:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:18:00.671009 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc62af8-c41f-4627-88ad-bcc0cb27d432 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:00.681642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc62af8-c41f-4627-88ad-bcc0cb27d432 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:00.687939 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24898167-0314-4a9f-8f31-ce57dec40570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:00.688611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24898167-0314-4a9f-8f31-ce57dec40570 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:00.688894 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24898167-0314-4a9f-8f31-ce57dec40570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:00.689170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24898167-0314-4a9f-8f31-ce57dec40570 admin admin] PATH_INFO: `/v3/projects/74fb83b3f039406b8b999b22b5ca8f3b/users/05f6ef74ae7549c787fd15a640675d8a/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:00.721199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24898167-0314-4a9f-8f31-ce57dec40570 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74fb83b3f039406b8b999b22b5ca8f3b, user_id=05f6ef74ae7549c787fd15a640675d8a, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:00.723965 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24898167-0314-4a9f-8f31-ce57dec40570 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:00.801696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1294/2609] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:18:00 2022] PUT /identity/v3/projects/74fb83b3f039406b8b999b22b5ca8f3b/users/05f6ef74ae7549c787fd15a640675d8a/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:00.805510 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4cab7-48e8-4a4d-a9d3-6ecabff6f658 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:00.824986 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4cab7-48e8-4a4d-a9d3-6ecabff6f658 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:00.890679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf7620-12d7-430f-8338-b90095efe5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:00.891335 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf7620-12d7-430f-8338-b90095efe5cf admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:00.891616 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf7620-12d7-430f-8338-b90095efe5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:00.892025 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf7620-12d7-430f-8338-b90095efe5cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:00.892669 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf7620-12d7-430f-8338-b90095efe5cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:00.894248 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf7620-12d7-430f-8338-b90095efe5cf admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:00.904409 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1316/2610] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:18:00 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:18:00.908620 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24898167-0314-4a9f-8f31-ce57dec40570 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:00.920161 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24898167-0314-4a9f-8f31-ce57dec40570 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:00.924801 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb4f09-a33a-4f73-b9e0-f70599177f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:00.925450 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb4f09-a33a-4f73-b9e0-f70599177f4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:00.925759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb4f09-a33a-4f73-b9e0-f70599177f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:00.926316 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb4f09-a33a-4f73-b9e0-f70599177f4a admin admin] PATH_INFO: `/v3/projects/74fb83b3f039406b8b999b22b5ca8f3b/users/05f6ef74ae7549c787fd15a640675d8a/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:00.928457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb4f09-a33a-4f73-b9e0-f70599177f4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=74fb83b3f039406b8b999b22b5ca8f3b, user_id=05f6ef74ae7549c787fd15a640675d8a, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:00.931620 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb4f09-a33a-4f73-b9e0-f70599177f4a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:00.952993 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1295/2611] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:18:00 2022] PUT /identity/v3/projects/74fb83b3f039406b8b999b22b5ca8f3b/users/05f6ef74ae7549c787fd15a640675d8a/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:00.954820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef1f15-afc9-4b45-b34c-e2a01755c3a3 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:00.955423 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef1f15-afc9-4b45-b34c-e2a01755c3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:00.955801 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef1f15-afc9-4b45-b34c-e2a01755c3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:00.964068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:00.988271 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-79ef1f15-afc9-4b45-b34c-e2a01755c3a3 None None] MFA Rules not processed for user `05f6ef74ae7549c787fd15a640675d8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:00.989810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-79ef1f15-afc9-4b45-b34c-e2a01755c3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:01.064248 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1317/2612] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:00 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:01.076756 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb4f09-a33a-4f73-b9e0-f70599177f4a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:01.078877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-35eb4f09-a33a-4f73-b9e0-f70599177f4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:01.140609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb4f09-a33a-4f73-b9e0-f70599177f4a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:01.144236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde2e49-257d-428c-b23d-5e2a1e8c26dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:01.144841 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde2e49-257d-428c-b23d-5e2a1e8c26dc service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:01.145122 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde2e49-257d-428c-b23d-5e2a1e8c26dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:01.145394 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde2e49-257d-428c-b23d-5e2a1e8c26dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:01.154840 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde2e49-257d-428c-b23d-5e2a1e8c26dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:01.156341 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde2e49-257d-428c-b23d-5e2a1e8c26dc service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:01.167541 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1296/2613] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:01 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:01.379577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef1f15-afc9-4b45-b34c-e2a01755c3a3 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:01.390111 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef1f15-afc9-4b45-b34c-e2a01755c3a3 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:01.393831 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c6db2-f057-4fd9-a4d2-9c0c21d4281a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:01.394428 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c6db2-f057-4fd9-a4d2-9c0c21d4281a service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:01.394711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c6db2-f057-4fd9-a4d2-9c0c21d4281a service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:01.394986 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c6db2-f057-4fd9-a4d2-9c0c21d4281a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:01.455250 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c6db2-f057-4fd9-a4d2-9c0c21d4281a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:01.456691 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c6db2-f057-4fd9-a4d2-9c0c21d4281a service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:01.465860 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1318/2614] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:02.096684 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde2e49-257d-428c-b23d-5e2a1e8c26dc service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:02.142141 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:02.159258 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde2e49-257d-428c-b23d-5e2a1e8c26dc service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:02.163125 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa481625-dd00-444f-b4f9-69db1ac46dcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:02.163840 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa481625-dd00-444f-b4f9-69db1ac46dcb service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:02.164112 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa481625-dd00-444f-b4f9-69db1ac46dcb service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:02.164385 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa481625-dd00-444f-b4f9-69db1ac46dcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:02.166146 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fa481625-dd00-444f-b4f9-69db1ac46dcb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:02.176889 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa481625-dd00-444f-b4f9-69db1ac46dcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:02.178312 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa481625-dd00-444f-b4f9-69db1ac46dcb service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:02.186377 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1297/2615] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:04.393719 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c6db2-f057-4fd9-a4d2-9c0c21d4281a service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:04.395123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ae2c6db2-f057-4fd9-a4d2-9c0c21d4281a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:04.459307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c6db2-f057-4fd9-a4d2-9c0c21d4281a service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:04.463942 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b48c9-b2bc-42ec-a7dc-3c3e374e234d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:04.464593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b48c9-b2bc-42ec-a7dc-3c3e374e234d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:04.464875 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b48c9-b2bc-42ec-a7dc-3c3e374e234d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:04.465150 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b48c9-b2bc-42ec-a7dc-3c3e374e234d admin admin] PATH_INFO: `/v3/users/c1bb19d59b2047f593f82fd6ca5d5e45` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:04.466282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09b48c9-b2bc-42ec-a7dc-3c3e374e234d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1bb19d59b2047f593f82fd6ca5d5e45)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:04.468264 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09b48c9-b2bc-42ec-a7dc-3c3e374e234d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:04.578666 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:04.702518 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e09b48c9-b2bc-42ec-a7dc-3c3e374e234d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1bb19d59b2047f593f82fd6ca5d5e45'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:04.710800 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e09b48c9-b2bc-42ec-a7dc-3c3e374e234d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1bb19d59b2047f593f82fd6ca5d5e45'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:04.799542 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e09b48c9-b2bc-42ec-a7dc-3c3e374e234d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1bb19d59b2047f593f82fd6ca5d5e45'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:04.800186 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e09b48c9-b2bc-42ec-a7dc-3c3e374e234d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1bb19d59b2047f593f82fd6ca5d5e45'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:04.800835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e09b48c9-b2bc-42ec-a7dc-3c3e374e234d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1bb19d59b2047f593f82fd6ca5d5e45'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:04.819706 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1319/2616] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:18:04 2022] DELETE /identity/v3/users/c1bb19d59b2047f593f82fd6ca5d5e45 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:04.824997 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f456a9c-1b9c-430f-a887-eb7b657af369 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:04.825484 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f456a9c-1b9c-430f-a887-eb7b657af369 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:04.825582 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f456a9c-1b9c-430f-a887-eb7b657af369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:04.845965 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9f456a9c-1b9c-430f-a887-eb7b657af369 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:04.847236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9f456a9c-1b9c-430f-a887-eb7b657af369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:04.907461 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1298/2617] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:04.917473 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b48c9-b2bc-42ec-a7dc-3c3e374e234d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:04.921608 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e09b48c9-b2bc-42ec-a7dc-3c3e374e234d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:04.977255 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b48c9-b2bc-42ec-a7dc-3c3e374e234d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:04.981181 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a60d8b-60e9-4e54-bee2-fe1966259210 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:04.984822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a60d8b-60e9-4e54-bee2-fe1966259210 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:04.984822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a60d8b-60e9-4e54-bee2-fe1966259210 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:04.984822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a60d8b-60e9-4e54-bee2-fe1966259210 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:04.992109 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a60d8b-60e9-4e54-bee2-fe1966259210 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:04.993530 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a60d8b-60e9-4e54-bee2-fe1966259210 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:05.004468 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1320/2618] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:05.385946 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f456a9c-1b9c-430f-a887-eb7b657af369 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:05.385946 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9f456a9c-1b9c-430f-a887-eb7b657af369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:05.398115 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f456a9c-1b9c-430f-a887-eb7b657af369 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:05.403722 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8787d093-2806-47bc-933a-5628a91f6010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:05.405450 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787d093-2806-47bc-933a-5628a91f6010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:05.405732 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787d093-2806-47bc-933a-5628a91f6010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:05.406010 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787d093-2806-47bc-933a-5628a91f6010 admin admin] PATH_INFO: `/v3/projects/8f23c11cbc914ab99f2ca12fcca3efc6` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:05.407204 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8787d093-2806-47bc-933a-5628a91f6010 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f23c11cbc914ab99f2ca12fcca3efc6)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:05.409129 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8787d093-2806-47bc-933a-5628a91f6010 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:05.508418 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8787d093-2806-47bc-933a-5628a91f6010 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f23c11cbc914ab99f2ca12fcca3efc6', 'request_id': 'req-8787d093-2806-47bc-933a-5628a91f6010'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:05.514913 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8787d093-2806-47bc-933a-5628a91f6010 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f23c11cbc914ab99f2ca12fcca3efc6', 'request_id': 'req-8787d093-2806-47bc-933a-5628a91f6010'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:05.576450 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8787d093-2806-47bc-933a-5628a91f6010 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f23c11cbc914ab99f2ca12fcca3efc6', 'request_id': 'req-8787d093-2806-47bc-933a-5628a91f6010'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:05.588070 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8787d093-2806-47bc-933a-5628a91f6010 admin admin] The token cache is being invalidate because project 8f23c11cbc914ab99f2ca12fcca3efc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:18:05.592043 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8787d093-2806-47bc-933a-5628a91f6010 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:05.592043 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8787d093-2806-47bc-933a-5628a91f6010 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:05.592043 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1299/2619] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:18:05 2022] DELETE /identity/v3/projects/8f23c11cbc914ab99f2ca12fcca3efc6 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:05.596009 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2ec5e-f02e-4ba1-9f9c-9d7381e2db81 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:05.596493 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2ec5e-f02e-4ba1-9f9c-9d7381e2db81 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:05.596594 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2ec5e-f02e-4ba1-9f9c-9d7381e2db81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:05.615906 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3cf2ec5e-f02e-4ba1-9f9c-9d7381e2db81 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:05.617109 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3cf2ec5e-f02e-4ba1-9f9c-9d7381e2db81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:05.674797 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1321/2620] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:05.682873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf89984-d9b5-432b-bc0c-37f4415a10d7 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:05.683166 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf89984-d9b5-432b-bc0c-37f4415a10d7 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:05.683464 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf89984-d9b5-432b-bc0c-37f4415a10d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:05.703794 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8bf89984-d9b5-432b-bc0c-37f4415a10d7 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:05.705044 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8bf89984-d9b5-432b-bc0c-37f4415a10d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:05.712855 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1300/2621] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:05.717238 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2ec5e-f02e-4ba1-9f9c-9d7381e2db81 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:05.727152 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2ec5e-f02e-4ba1-9f9c-9d7381e2db81 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:05.732218 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe9d3d-3a0a-474b-a062-07e599355db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:05.732812 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe9d3d-3a0a-474b-a062-07e599355db8 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:05.733114 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe9d3d-3a0a-474b-a062-07e599355db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:05.733383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe9d3d-3a0a-474b-a062-07e599355db8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:05.734136 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fe9d3d-3a0a-474b-a062-07e599355db8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:18:05.734468 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fe9d3d-3a0a-474b-a062-07e599355db8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:05.736167 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fe9d3d-3a0a-474b-a062-07e599355db8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:05.748889 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1322/2622] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 13:18:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:18:05.753134 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf89984-d9b5-432b-bc0c-37f4415a10d7 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:05.763189 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf89984-d9b5-432b-bc0c-37f4415a10d7 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:05.767860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce349d3-3d97-4eb5-ad07-866fb5c2b7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:05.768446 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce349d3-3d97-4eb5-ad07-866fb5c2b7c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:05.768726 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce349d3-3d97-4eb5-ad07-866fb5c2b7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:05.768992 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce349d3-3d97-4eb5-ad07-866fb5c2b7c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:05.769607 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce349d3-3d97-4eb5-ad07-866fb5c2b7c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:05.771526 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce349d3-3d97-4eb5-ad07-866fb5c2b7c4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:05.858974 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1301/2623] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:18:05 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:18:05.862633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe9d3d-3a0a-474b-a062-07e599355db8 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:05.873144 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe9d3d-3a0a-474b-a062-07e599355db8 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:05.905262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:05.929303 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182f156-bdd5-41d6-b6d2-a910ac38b77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:05.929825 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182f156-bdd5-41d6-b6d2-a910ac38b77c admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:05.930108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182f156-bdd5-41d6-b6d2-a910ac38b77c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:05.930384 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182f156-bdd5-41d6-b6d2-a910ac38b77c admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:05.931010 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6182f156-bdd5-41d6-b6d2-a910ac38b77c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:05.933630 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6182f156-bdd5-41d6-b6d2-a910ac38b77c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:06.010855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1323/2624] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:18:05 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:18:06.014943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce349d3-3d97-4eb5-ad07-866fb5c2b7c4 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:06.025610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce349d3-3d97-4eb5-ad07-866fb5c2b7c4 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:06.030457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55395038-247a-49b4-8f3a-36c66cd1092e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:06.031049 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55395038-247a-49b4-8f3a-36c66cd1092e admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:06.032943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55395038-247a-49b4-8f3a-36c66cd1092e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:06.032943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55395038-247a-49b4-8f3a-36c66cd1092e admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:06.032943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55395038-247a-49b4-8f3a-36c66cd1092e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:06.033885 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55395038-247a-49b4-8f3a-36c66cd1092e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:06.044861 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1302/2625] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:18:06.050180 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182f156-bdd5-41d6-b6d2-a910ac38b77c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:06.059568 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182f156-bdd5-41d6-b6d2-a910ac38b77c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:06.064176 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86b5a5-6a1c-44fe-ac20-4e9b01e6c019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:06.064757 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86b5a5-6a1c-44fe-ac20-4e9b01e6c019 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:06.065034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86b5a5-6a1c-44fe-ac20-4e9b01e6c019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:06.065302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86b5a5-6a1c-44fe-ac20-4e9b01e6c019 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:06.065928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac86b5a5-6a1c-44fe-ac20-4e9b01e6c019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:06.067545 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac86b5a5-6a1c-44fe-ac20-4e9b01e6c019 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:06.081489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1324/2626] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:18:06.082772 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55395038-247a-49b4-8f3a-36c66cd1092e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:06.093617 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55395038-247a-49b4-8f3a-36c66cd1092e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:06.098937 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bda33e-e69c-4f09-b94e-dcb2ce7eab93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:06.099743 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bda33e-e69c-4f09-b94e-dcb2ce7eab93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:06.100026 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bda33e-e69c-4f09-b94e-dcb2ce7eab93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:06.100300 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bda33e-e69c-4f09-b94e-dcb2ce7eab93 admin admin] PATH_INFO: `/v3/projects/2a1915ca912a42b88145ea1bf2dbc049/users/41368bdb7a1648848f483eb4a12e3c38/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:06.123369 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bda33e-e69c-4f09-b94e-dcb2ce7eab93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a1915ca912a42b88145ea1bf2dbc049, user_id=41368bdb7a1648848f483eb4a12e3c38, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:06.126204 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bda33e-e69c-4f09-b94e-dcb2ce7eab93 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:06.200223 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:06.205770 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1303/2627] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:18:06 2022] PUT /identity/v3/projects/2a1915ca912a42b88145ea1bf2dbc049/users/41368bdb7a1648848f483eb4a12e3c38/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:06.217876 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe88f2-49b8-4954-bc9e-42161f60febe None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:06.228933 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe88f2-49b8-4954-bc9e-42161f60febe None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:06.228933 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe88f2-49b8-4954-bc9e-42161f60febe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:06.250386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-10fe88f2-49b8-4954-bc9e-42161f60febe None None] MFA Rules not processed for user `41368bdb7a1648848f483eb4a12e3c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:06.251623 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-10fe88f2-49b8-4954-bc9e-42161f60febe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:06.318880 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1325/2628] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:06.330019 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bda33e-e69c-4f09-b94e-dcb2ce7eab93 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:06.331803 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-91bda33e-e69c-4f09-b94e-dcb2ce7eab93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:06.390095 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bda33e-e69c-4f09-b94e-dcb2ce7eab93 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:06.393882 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304fedac-33d0-42a0-8435-c32ef27ecee7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:06.394485 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304fedac-33d0-42a0-8435-c32ef27ecee7 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:06.394769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304fedac-33d0-42a0-8435-c32ef27ecee7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:06.395043 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304fedac-33d0-42a0-8435-c32ef27ecee7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:06.403484 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304fedac-33d0-42a0-8435-c32ef27ecee7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:06.404917 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304fedac-33d0-42a0-8435-c32ef27ecee7 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:06.413629 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1304/2629] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:09.193436 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493147d-f621-444c-89f5-200355454aba None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:09.193879 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493147d-f621-444c-89f5-200355454aba None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:09.193994 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493147d-f621-444c-89f5-200355454aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:09.215217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6493147d-f621-444c-89f5-200355454aba None None] MFA Rules not processed for user `41368bdb7a1648848f483eb4a12e3c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:09.216446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6493147d-f621-444c-89f5-200355454aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:09.223965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1326/2630] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:12.957149 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304fedac-33d0-42a0-8435-c32ef27ecee7 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:12.958679 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-304fedac-33d0-42a0-8435-c32ef27ecee7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:13.026893 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304fedac-33d0-42a0-8435-c32ef27ecee7 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:13.031737 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6704d0fb-4197-45dc-a608-bbc212b4b1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:13.032369 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704d0fb-4197-45dc-a608-bbc212b4b1f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:13.032651 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704d0fb-4197-45dc-a608-bbc212b4b1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:13.032932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704d0fb-4197-45dc-a608-bbc212b4b1f8 admin admin] PATH_INFO: `/v3/users/51bbcebbad284b2fb846eeb3e186ed2b` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:13.034096 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6704d0fb-4197-45dc-a608-bbc212b4b1f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51bbcebbad284b2fb846eeb3e186ed2b)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:13.036058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6704d0fb-4197-45dc-a608-bbc212b4b1f8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:13.255803 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6704d0fb-4197-45dc-a608-bbc212b4b1f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51bbcebbad284b2fb846eeb3e186ed2b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:13.315258 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6704d0fb-4197-45dc-a608-bbc212b4b1f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51bbcebbad284b2fb846eeb3e186ed2b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:13.316073 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6704d0fb-4197-45dc-a608-bbc212b4b1f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51bbcebbad284b2fb846eeb3e186ed2b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:13.316822 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6704d0fb-4197-45dc-a608-bbc212b4b1f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51bbcebbad284b2fb846eeb3e186ed2b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:13.328007 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6704d0fb-4197-45dc-a608-bbc212b4b1f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51bbcebbad284b2fb846eeb3e186ed2b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:13.357432 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1305/2631] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:18:12 2022] DELETE /identity/v3/users/51bbcebbad284b2fb846eeb3e186ed2b => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:13.359291 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2488f-4fa0-4fb0-9026-5ed6b6a8e52b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:13.359771 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2488f-4fa0-4fb0-9026-5ed6b6a8e52b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:13.359878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2488f-4fa0-4fb0-9026-5ed6b6a8e52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:13.388388 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-60d2488f-4fa0-4fb0-9026-5ed6b6a8e52b None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:13.389606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-60d2488f-4fa0-4fb0-9026-5ed6b6a8e52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:13.464298 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1327/2632] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:18:13.477838 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6704d0fb-4197-45dc-a608-bbc212b4b1f8 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:13.477838 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6704d0fb-4197-45dc-a608-bbc212b4b1f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:13.506225 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:13.547437 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6704d0fb-4197-45dc-a608-bbc212b4b1f8 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:13.551027 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b538fd2-6eae-4d30-bca1-0cf6cb094522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:13.551674 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b538fd2-6eae-4d30-bca1-0cf6cb094522 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:13.551957 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b538fd2-6eae-4d30-bca1-0cf6cb094522 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:13.552237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b538fd2-6eae-4d30-bca1-0cf6cb094522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:13.561321 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b538fd2-6eae-4d30-bca1-0cf6cb094522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:13.562755 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b538fd2-6eae-4d30-bca1-0cf6cb094522 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:13.572446 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1306/2633] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:14.077681 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d2488f-4fa0-4fb0-9026-5ed6b6a8e52b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:14.079264 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-60d2488f-4fa0-4fb0-9026-5ed6b6a8e52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:14.094897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d2488f-4fa0-4fb0-9026-5ed6b6a8e52b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:14.100319 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb89884-0345-4a5c-98be-1cc5967f0ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:14.101037 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb89884-0345-4a5c-98be-1cc5967f0ed0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:14.101336 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb89884-0345-4a5c-98be-1cc5967f0ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:14.101635 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb89884-0345-4a5c-98be-1cc5967f0ed0 admin admin] PATH_INFO: `/v3/projects/fdf1111675a94fdc9685b4447bb2cb66` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:14.102776 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb89884-0345-4a5c-98be-1cc5967f0ed0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdf1111675a94fdc9685b4447bb2cb66)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:14.106813 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb89884-0345-4a5c-98be-1cc5967f0ed0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:14.198371 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:14.211580 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2bb89884-0345-4a5c-98be-1cc5967f0ed0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdf1111675a94fdc9685b4447bb2cb66', 'request_id': 'req-2bb89884-0345-4a5c-98be-1cc5967f0ed0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:14.219338 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2bb89884-0345-4a5c-98be-1cc5967f0ed0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdf1111675a94fdc9685b4447bb2cb66', 'request_id': 'req-2bb89884-0345-4a5c-98be-1cc5967f0ed0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:14.297363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2bb89884-0345-4a5c-98be-1cc5967f0ed0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdf1111675a94fdc9685b4447bb2cb66', 'request_id': 'req-2bb89884-0345-4a5c-98be-1cc5967f0ed0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:14.309828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2bb89884-0345-4a5c-98be-1cc5967f0ed0 admin admin] The token cache is being invalidate because project fdf1111675a94fdc9685b4447bb2cb66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:18:14.310335 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2bb89884-0345-4a5c-98be-1cc5967f0ed0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:14.310998 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2bb89884-0345-4a5c-98be-1cc5967f0ed0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:14.312524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1328/2634] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:18:14 2022] DELETE /identity/v3/projects/fdf1111675a94fdc9685b4447bb2cb66 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:14.316498 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b538fd2-6eae-4d30-bca1-0cf6cb094522 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:14.326986 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b538fd2-6eae-4d30-bca1-0cf6cb094522 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:14.386520 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac24722-edd3-421a-92d6-011d1a53ff22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:14.387162 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac24722-edd3-421a-92d6-011d1a53ff22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:14.387472 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac24722-edd3-421a-92d6-011d1a53ff22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:14.387749 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac24722-edd3-421a-92d6-011d1a53ff22 admin admin] PATH_INFO: `/v3/users/05f6ef74ae7549c787fd15a640675d8a` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:14.388881 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac24722-edd3-421a-92d6-011d1a53ff22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05f6ef74ae7549c787fd15a640675d8a)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:14.390789 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac24722-edd3-421a-92d6-011d1a53ff22 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:14.544126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-aac24722-edd3-421a-92d6-011d1a53ff22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05f6ef74ae7549c787fd15a640675d8a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:14.588710 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-aac24722-edd3-421a-92d6-011d1a53ff22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05f6ef74ae7549c787fd15a640675d8a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:14.590109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-aac24722-edd3-421a-92d6-011d1a53ff22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05f6ef74ae7549c787fd15a640675d8a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:14.590872 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-aac24722-edd3-421a-92d6-011d1a53ff22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05f6ef74ae7549c787fd15a640675d8a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:14.598325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-aac24722-edd3-421a-92d6-011d1a53ff22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05f6ef74ae7549c787fd15a640675d8a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:14.616938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1307/2635] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:18:14 2022] DELETE /identity/v3/users/05f6ef74ae7549c787fd15a640675d8a => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:15.121507 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb89884-0345-4a5c-98be-1cc5967f0ed0 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:15.123057 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2bb89884-0345-4a5c-98be-1cc5967f0ed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:15.136364 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb89884-0345-4a5c-98be-1cc5967f0ed0 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:15.200857 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59ff9b-354e-45af-9373-b97e00e2823e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:15.201494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59ff9b-354e-45af-9373-b97e00e2823e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:15.201777 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59ff9b-354e-45af-9373-b97e00e2823e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:15.202051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59ff9b-354e-45af-9373-b97e00e2823e admin admin] PATH_INFO: `/v3/projects/74fb83b3f039406b8b999b22b5ca8f3b` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:15.203193 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59ff9b-354e-45af-9373-b97e00e2823e admin admin] RBAC: Authorizing `identity:delete_project(project_id=74fb83b3f039406b8b999b22b5ca8f3b)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:15.205112 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59ff9b-354e-45af-9373-b97e00e2823e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:15.308433 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9a59ff9b-354e-45af-9373-b97e00e2823e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74fb83b3f039406b8b999b22b5ca8f3b', 'request_id': 'req-9a59ff9b-354e-45af-9373-b97e00e2823e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:15.314535 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9a59ff9b-354e-45af-9373-b97e00e2823e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74fb83b3f039406b8b999b22b5ca8f3b', 'request_id': 'req-9a59ff9b-354e-45af-9373-b97e00e2823e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:15.359354 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9a59ff9b-354e-45af-9373-b97e00e2823e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74fb83b3f039406b8b999b22b5ca8f3b', 'request_id': 'req-9a59ff9b-354e-45af-9373-b97e00e2823e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:15.373271 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9a59ff9b-354e-45af-9373-b97e00e2823e admin admin] The token cache is being invalidate because project 74fb83b3f039406b8b999b22b5ca8f3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:18:15.373519 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9a59ff9b-354e-45af-9373-b97e00e2823e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:15.374181 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9a59ff9b-354e-45af-9373-b97e00e2823e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:15.375729 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1329/2636] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:18:15 2022] DELETE /identity/v3/projects/74fb83b3f039406b8b999b22b5ca8f3b => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:15.380538 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939235e0-5432-42bf-8153-b9e838ee9cfe None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:15.381044 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939235e0-5432-42bf-8153-b9e838ee9cfe None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:15.384502 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939235e0-5432-42bf-8153-b9e838ee9cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:15.400943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-939235e0-5432-42bf-8153-b9e838ee9cfe None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:15.402204 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-939235e0-5432-42bf-8153-b9e838ee9cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:15.440051 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:15.460263 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1308/2637] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:15.469694 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146811ef-93e0-49d6-9a05-e3c4ea4faa5b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:15.469974 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146811ef-93e0-49d6-9a05-e3c4ea4faa5b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:15.470243 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146811ef-93e0-49d6-9a05-e3c4ea4faa5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:15.490226 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-146811ef-93e0-49d6-9a05-e3c4ea4faa5b None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:15.491460 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-146811ef-93e0-49d6-9a05-e3c4ea4faa5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:15.500129 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1330/2638] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:15.504819 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939235e0-5432-42bf-8153-b9e838ee9cfe None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:15.515297 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939235e0-5432-42bf-8153-b9e838ee9cfe None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:15.520423 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b76fd6-6825-4519-8d99-d22b795f8f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:15.521009 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b76fd6-6825-4519-8d99-d22b795f8f84 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:15.521287 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b76fd6-6825-4519-8d99-d22b795f8f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:15.521556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b76fd6-6825-4519-8d99-d22b795f8f84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:15.522195 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b76fd6-6825-4519-8d99-d22b795f8f84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:18:15.522501 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b76fd6-6825-4519-8d99-d22b795f8f84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:15.524107 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b76fd6-6825-4519-8d99-d22b795f8f84 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:15.535545 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1309/2639] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 13:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:18:15.539772 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146811ef-93e0-49d6-9a05-e3c4ea4faa5b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:15.549144 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146811ef-93e0-49d6-9a05-e3c4ea4faa5b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:15.553792 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c94f8-1f92-4e58-bafc-c36f8df66393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:15.554385 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c94f8-1f92-4e58-bafc-c36f8df66393 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:15.554664 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c94f8-1f92-4e58-bafc-c36f8df66393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:15.554939 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c94f8-1f92-4e58-bafc-c36f8df66393 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:15.555586 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124c94f8-1f92-4e58-bafc-c36f8df66393 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:15.557454 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124c94f8-1f92-4e58-bafc-c36f8df66393 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:15.629514 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1331/2640] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:18:15 2022] POST /identity/v3/projects => generated 367 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:18:15.633497 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b76fd6-6825-4519-8d99-d22b795f8f84 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:15.644137 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b76fd6-6825-4519-8d99-d22b795f8f84 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:15.700619 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cbb4aa-52ae-4e3c-b511-cc03652e2c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:15.701225 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbb4aa-52ae-4e3c-b511-cc03652e2c6a admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:15.701507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbb4aa-52ae-4e3c-b511-cc03652e2c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:15.701804 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbb4aa-52ae-4e3c-b511-cc03652e2c6a admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:15.702432 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cbb4aa-52ae-4e3c-b511-cc03652e2c6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:15.704372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cbb4aa-52ae-4e3c-b511-cc03652e2c6a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:15.789800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1310/2641] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:18:15 2022] POST /identity/v3/users => generated 284 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:18:15.794626 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c94f8-1f92-4e58-bafc-c36f8df66393 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:15.805894 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c94f8-1f92-4e58-bafc-c36f8df66393 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:15.811726 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e260d-601c-4feb-99d7-e2f08d6e787e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:15.812404 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e260d-601c-4feb-99d7-e2f08d6e787e admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:15.812783 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e260d-601c-4feb-99d7-e2f08d6e787e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:15.813145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e260d-601c-4feb-99d7-e2f08d6e787e admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:15.813860 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5e260d-601c-4feb-99d7-e2f08d6e787e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:15.815569 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5e260d-601c-4feb-99d7-e2f08d6e787e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:15.833067 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1332/2642] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:18:15.837176 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cbb4aa-52ae-4e3c-b511-cc03652e2c6a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:15.848098 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cbb4aa-52ae-4e3c-b511-cc03652e2c6a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:15.853020 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93fc91-efa6-4e46-a57b-68bac89ab6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:15.853622 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93fc91-efa6-4e46-a57b-68bac89ab6c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:15.853902 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93fc91-efa6-4e46-a57b-68bac89ab6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:15.854206 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93fc91-efa6-4e46-a57b-68bac89ab6c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:15.855123 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af93fc91-efa6-4e46-a57b-68bac89ab6c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:15.857273 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af93fc91-efa6-4e46-a57b-68bac89ab6c7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:15.869157 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1311/2643] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:18:15.874254 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e260d-601c-4feb-99d7-e2f08d6e787e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:15.884486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e260d-601c-4feb-99d7-e2f08d6e787e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:15.889298 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9bd010-72f1-463a-915b-7704c9e6dc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:15.889954 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9bd010-72f1-463a-915b-7704c9e6dc84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:15.890232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9bd010-72f1-463a-915b-7704c9e6dc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:15.890505 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9bd010-72f1-463a-915b-7704c9e6dc84 admin admin] PATH_INFO: `/v3/projects/8a5c99f73ac14b68892ea8a83c503c60/users/05828d9bf3ad4cd381337e29381cf590/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:15.899090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:15.906352 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9bd010-72f1-463a-915b-7704c9e6dc84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a5c99f73ac14b68892ea8a83c503c60, user_id=05828d9bf3ad4cd381337e29381cf590, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:15.909141 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9bd010-72f1-463a-915b-7704c9e6dc84 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:16.000205 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1333/2644] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:18:15 2022] PUT /identity/v3/projects/8a5c99f73ac14b68892ea8a83c503c60/users/05828d9bf3ad4cd381337e29381cf590/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:16.010130 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b6846-ad53-46a1-88af-756b50e92f3b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:16.010419 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b6846-ad53-46a1-88af-756b50e92f3b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:16.010696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b6846-ad53-46a1-88af-756b50e92f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:16.037995 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-fd6b6846-ad53-46a1-88af-756b50e92f3b None None] MFA Rules not processed for user `05828d9bf3ad4cd381337e29381cf590`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:16.039247 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fd6b6846-ad53-46a1-88af-756b50e92f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:16.101631 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1312/2645] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:16.106744 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9bd010-72f1-463a-915b-7704c9e6dc84 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:16.172403 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9bd010-72f1-463a-915b-7704c9e6dc84 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:16.177002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de873e-8e5e-480a-a4e5-ed5ccaa22f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:16.177609 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de873e-8e5e-480a-a4e5-ed5ccaa22f58 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:16.177887 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de873e-8e5e-480a-a4e5-ed5ccaa22f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:16.178159 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de873e-8e5e-480a-a4e5-ed5ccaa22f58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:16.178774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de873e-8e5e-480a-a4e5-ed5ccaa22f58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:16.180978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de873e-8e5e-480a-a4e5-ed5ccaa22f58 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:16.279606 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1334/2646] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:18:16 2022] POST /identity/v3/projects => generated 367 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:18:16.280994 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6b6846-ad53-46a1-88af-756b50e92f3b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:16.291881 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6b6846-ad53-46a1-88af-756b50e92f3b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:16.324648 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:16.347219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc293161-8945-47f8-9fd8-2ebc6597536d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:16.347824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc293161-8945-47f8-9fd8-2ebc6597536d admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:16.348103 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc293161-8945-47f8-9fd8-2ebc6597536d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:16.348373 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc293161-8945-47f8-9fd8-2ebc6597536d admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:16.348994 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc293161-8945-47f8-9fd8-2ebc6597536d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:16.350877 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc293161-8945-47f8-9fd8-2ebc6597536d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:16.445080 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1313/2647] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:18:16 2022] POST /identity/v3/users => generated 290 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:18:16.448952 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de873e-8e5e-480a-a4e5-ed5ccaa22f58 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:16.459730 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de873e-8e5e-480a-a4e5-ed5ccaa22f58 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:16.464301 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce75bfe-578b-4f75-b198-ce75c15b99a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:16.464892 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce75bfe-578b-4f75-b198-ce75c15b99a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:16.465167 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce75bfe-578b-4f75-b198-ce75c15b99a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:16.465442 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce75bfe-578b-4f75-b198-ce75c15b99a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:16.466760 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce75bfe-578b-4f75-b198-ce75c15b99a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:16.468381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce75bfe-578b-4f75-b198-ce75c15b99a3 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:16.478874 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1335/2648] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:18:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:18:16.484410 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc293161-8945-47f8-9fd8-2ebc6597536d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:16.494551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc293161-8945-47f8-9fd8-2ebc6597536d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:16.499646 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eccd25f-8a69-40a8-9dc4-8175401baa3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:16.500287 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eccd25f-8a69-40a8-9dc4-8175401baa3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:16.500564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eccd25f-8a69-40a8-9dc4-8175401baa3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:16.500837 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eccd25f-8a69-40a8-9dc4-8175401baa3a admin admin] PATH_INFO: `/v3/projects/7ed555def50d4682a7eafb00d0acb8fb/users/70e00b395e1a48b3ab4ee036669a067f/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:16.516276 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eccd25f-8a69-40a8-9dc4-8175401baa3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ed555def50d4682a7eafb00d0acb8fb, user_id=70e00b395e1a48b3ab4ee036669a067f, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:16.518890 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eccd25f-8a69-40a8-9dc4-8175401baa3a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:16.577199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1314/2649] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:18:16 2022] PUT /identity/v3/projects/7ed555def50d4682a7eafb00d0acb8fb/users/70e00b395e1a48b3ab4ee036669a067f/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:16.581070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce75bfe-578b-4f75-b198-ce75c15b99a3 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:16.591674 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce75bfe-578b-4f75-b198-ce75c15b99a3 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:16.648537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa771a8-503c-4d9c-98b5-9290286e1ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:16.649413 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa771a8-503c-4d9c-98b5-9290286e1ef9 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:16.649686 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa771a8-503c-4d9c-98b5-9290286e1ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:16.649951 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa771a8-503c-4d9c-98b5-9290286e1ef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:16.650579 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa771a8-503c-4d9c-98b5-9290286e1ef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:16.652178 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa771a8-503c-4d9c-98b5-9290286e1ef9 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:16.662906 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1336/2650] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:18:16 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:18:16.666954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eccd25f-8a69-40a8-9dc4-8175401baa3a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:16.677860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eccd25f-8a69-40a8-9dc4-8175401baa3a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:16.682420 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f8c60d-0c46-4ac7-b605-9baea9b096bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:16.683063 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8c60d-0c46-4ac7-b605-9baea9b096bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:16.683379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8c60d-0c46-4ac7-b605-9baea9b096bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:16.683655 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8c60d-0c46-4ac7-b605-9baea9b096bd admin admin] PATH_INFO: `/v3/projects/7ed555def50d4682a7eafb00d0acb8fb/users/70e00b395e1a48b3ab4ee036669a067f/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:16.685667 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f8c60d-0c46-4ac7-b605-9baea9b096bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ed555def50d4682a7eafb00d0acb8fb, user_id=70e00b395e1a48b3ab4ee036669a067f, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:16.688330 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f8c60d-0c46-4ac7-b605-9baea9b096bd admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:16.708015 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1315/2651] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:18:16 2022] PUT /identity/v3/projects/7ed555def50d4682a7eafb00d0acb8fb/users/70e00b395e1a48b3ab4ee036669a067f/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:16.710810 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852bb51-8055-4cd1-94ab-c176dadccfcf None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:16.711101 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852bb51-8055-4cd1-94ab-c176dadccfcf None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:16.711413 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852bb51-8055-4cd1-94ab-c176dadccfcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:16.742038 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7852bb51-8055-4cd1-94ab-c176dadccfcf None None] MFA Rules not processed for user `70e00b395e1a48b3ab4ee036669a067f`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:16.743520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7852bb51-8055-4cd1-94ab-c176dadccfcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:16.791765 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:16.816583 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1337/2652] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:16 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:16.829295 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f8c60d-0c46-4ac7-b605-9baea9b096bd admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:16.831333 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-92f8c60d-0c46-4ac7-b605-9baea9b096bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:16.924170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f8c60d-0c46-4ac7-b605-9baea9b096bd admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:16.928351 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b983144-78f2-4e50-8480-cc040d0f06da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:16.928974 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b983144-78f2-4e50-8480-cc040d0f06da service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:16.931140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b983144-78f2-4e50-8480-cc040d0f06da service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:16.931464 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b983144-78f2-4e50-8480-cc040d0f06da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:16.945920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b983144-78f2-4e50-8480-cc040d0f06da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:16.947738 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b983144-78f2-4e50-8480-cc040d0f06da service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:16.961166 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1316/2653] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:16 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:19.624066 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852bb51-8055-4cd1-94ab-c176dadccfcf None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:19.688019 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852bb51-8055-4cd1-94ab-c176dadccfcf None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:19.692620 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7b78f-4401-4868-b392-a7d750f03504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:19.693375 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7b78f-4401-4868-b392-a7d750f03504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:19.693572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7b78f-4401-4868-b392-a7d750f03504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:19.693833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7b78f-4401-4868-b392-a7d750f03504 admin admin] PATH_INFO: `/v3/users/05828d9bf3ad4cd381337e29381cf590` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:19.695216 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d7b78f-4401-4868-b392-a7d750f03504 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05828d9bf3ad4cd381337e29381cf590)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:19.697136 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d7b78f-4401-4868-b392-a7d750f03504 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:19.928193 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-30d7b78f-4401-4868-b392-a7d750f03504 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05828d9bf3ad4cd381337e29381cf590'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:19.935693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-30d7b78f-4401-4868-b392-a7d750f03504 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05828d9bf3ad4cd381337e29381cf590'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:20.010676 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-30d7b78f-4401-4868-b392-a7d750f03504 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05828d9bf3ad4cd381337e29381cf590'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:20.011326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-30d7b78f-4401-4868-b392-a7d750f03504 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05828d9bf3ad4cd381337e29381cf590'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:20.012008 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-30d7b78f-4401-4868-b392-a7d750f03504 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05828d9bf3ad4cd381337e29381cf590'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:20.030192 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1338/2654] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:18:19 2022] DELETE /identity/v3/users/05828d9bf3ad4cd381337e29381cf590 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:20.034903 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d4699-d148-4162-9ffd-f7bce78c6858 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:20.035400 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d4699-d148-4162-9ffd-f7bce78c6858 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:20.035489 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d4699-d148-4162-9ffd-f7bce78c6858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:20.054930 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-519d4699-d148-4162-9ffd-f7bce78c6858 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:20.056183 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-519d4699-d148-4162-9ffd-f7bce78c6858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:20.115430 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1317/2655] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:20.125656 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7b78f-4401-4868-b392-a7d750f03504 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:20.127060 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-30d7b78f-4401-4868-b392-a7d750f03504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:20.184602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7b78f-4401-4868-b392-a7d750f03504 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:20.188401 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3c6f3-50f3-47bc-9af8-ebdac45716a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:20.189011 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3c6f3-50f3-47bc-9af8-ebdac45716a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:20.189293 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3c6f3-50f3-47bc-9af8-ebdac45716a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:20.189591 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3c6f3-50f3-47bc-9af8-ebdac45716a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:20.199005 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3c6f3-50f3-47bc-9af8-ebdac45716a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:20.201034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3c6f3-50f3-47bc-9af8-ebdac45716a1 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:20.210593 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1339/2656] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:20.579314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d4699-d148-4162-9ffd-f7bce78c6858 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:20.580757 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-519d4699-d148-4162-9ffd-f7bce78c6858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:20.593739 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d4699-d148-4162-9ffd-f7bce78c6858 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:20.600770 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ae90a-4063-40fe-b3a1-97070c4e7bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:20.601605 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ae90a-4063-40fe-b3a1-97070c4e7bc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:20.602039 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ae90a-4063-40fe-b3a1-97070c4e7bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:20.602312 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ae90a-4063-40fe-b3a1-97070c4e7bc3 admin admin] PATH_INFO: `/v3/projects/8a5c99f73ac14b68892ea8a83c503c60` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:20.603486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ae90a-4063-40fe-b3a1-97070c4e7bc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a5c99f73ac14b68892ea8a83c503c60)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:20.605687 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ae90a-4063-40fe-b3a1-97070c4e7bc3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:20.705377 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:20.723168 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ac0ae90a-4063-40fe-b3a1-97070c4e7bc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a5c99f73ac14b68892ea8a83c503c60', 'request_id': 'req-ac0ae90a-4063-40fe-b3a1-97070c4e7bc3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:20.728867 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ac0ae90a-4063-40fe-b3a1-97070c4e7bc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a5c99f73ac14b68892ea8a83c503c60', 'request_id': 'req-ac0ae90a-4063-40fe-b3a1-97070c4e7bc3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:20.816809 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ac0ae90a-4063-40fe-b3a1-97070c4e7bc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a5c99f73ac14b68892ea8a83c503c60', 'request_id': 'req-ac0ae90a-4063-40fe-b3a1-97070c4e7bc3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:20.829602 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ac0ae90a-4063-40fe-b3a1-97070c4e7bc3 admin admin] The token cache is being invalidate because project 8a5c99f73ac14b68892ea8a83c503c60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:18:20.829973 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ac0ae90a-4063-40fe-b3a1-97070c4e7bc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:20.830633 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ac0ae90a-4063-40fe-b3a1-97070c4e7bc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:20.832327 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1318/2657] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:18:20 2022] DELETE /identity/v3/projects/8a5c99f73ac14b68892ea8a83c503c60 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:20.836159 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3c6f3-50f3-47bc-9af8-ebdac45716a1 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:20.842783 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:20.846878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3c6f3-50f3-47bc-9af8-ebdac45716a1 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:20.909095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a453a-338b-4d02-8e4e-807303dfd0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:20.909701 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a453a-338b-4d02-8e4e-807303dfd0e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:20.910248 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a453a-338b-4d02-8e4e-807303dfd0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:20.910521 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a453a-338b-4d02-8e4e-807303dfd0e4 admin admin] PATH_INFO: `/v3/users/70e00b395e1a48b3ab4ee036669a067f` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:20.911682 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a453a-338b-4d02-8e4e-807303dfd0e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70e00b395e1a48b3ab4ee036669a067f)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:20.913695 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a453a-338b-4d02-8e4e-807303dfd0e4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:21.075983 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b55a453a-338b-4d02-8e4e-807303dfd0e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70e00b395e1a48b3ab4ee036669a067f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:21.083359 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b55a453a-338b-4d02-8e4e-807303dfd0e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70e00b395e1a48b3ab4ee036669a067f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:21.171011 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b55a453a-338b-4d02-8e4e-807303dfd0e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70e00b395e1a48b3ab4ee036669a067f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:21.171611 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b55a453a-338b-4d02-8e4e-807303dfd0e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70e00b395e1a48b3ab4ee036669a067f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:21.172231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b55a453a-338b-4d02-8e4e-807303dfd0e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70e00b395e1a48b3ab4ee036669a067f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:21.190872 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1340/2658] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:18:20 2022] DELETE /identity/v3/users/70e00b395e1a48b3ab4ee036669a067f => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:21.550177 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ae90a-4063-40fe-b3a1-97070c4e7bc3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:21.551658 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ac0ae90a-4063-40fe-b3a1-97070c4e7bc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:21.565519 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ae90a-4063-40fe-b3a1-97070c4e7bc3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:21.624425 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d99df-ee2a-4025-afc0-2f1a7492f70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:21.625052 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d99df-ee2a-4025-afc0-2f1a7492f70b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:21.625328 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d99df-ee2a-4025-afc0-2f1a7492f70b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:21.625599 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d99df-ee2a-4025-afc0-2f1a7492f70b admin admin] PATH_INFO: `/v3/projects/7ed555def50d4682a7eafb00d0acb8fb` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:21.626705 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d99df-ee2a-4025-afc0-2f1a7492f70b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ed555def50d4682a7eafb00d0acb8fb)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:21.628625 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d99df-ee2a-4025-afc0-2f1a7492f70b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:21.708354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e84d99df-ee2a-4025-afc0-2f1a7492f70b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ed555def50d4682a7eafb00d0acb8fb', 'request_id': 'req-e84d99df-ee2a-4025-afc0-2f1a7492f70b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:21.713760 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e84d99df-ee2a-4025-afc0-2f1a7492f70b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ed555def50d4682a7eafb00d0acb8fb', 'request_id': 'req-e84d99df-ee2a-4025-afc0-2f1a7492f70b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:21.811533 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e84d99df-ee2a-4025-afc0-2f1a7492f70b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ed555def50d4682a7eafb00d0acb8fb', 'request_id': 'req-e84d99df-ee2a-4025-afc0-2f1a7492f70b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:21.823685 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e84d99df-ee2a-4025-afc0-2f1a7492f70b admin admin] The token cache is being invalidate because project 7ed555def50d4682a7eafb00d0acb8fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:18:21.823977 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e84d99df-ee2a-4025-afc0-2f1a7492f70b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:21.824666 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e84d99df-ee2a-4025-afc0-2f1a7492f70b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:21.826244 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1319/2659] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:18:21 2022] DELETE /identity/v3/projects/7ed555def50d4682a7eafb00d0acb8fb => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:21.831137 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9f20d-9c9f-4828-bc4d-9c5cfd2e1f63 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:21.831534 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9f20d-9c9f-4828-bc4d-9c5cfd2e1f63 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:21.831876 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9f20d-9c9f-4828-bc4d-9c5cfd2e1f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:21.852079 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6ad9f20d-9c9f-4828-bc4d-9c5cfd2e1f63 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:21.853333 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6ad9f20d-9c9f-4828-bc4d-9c5cfd2e1f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:21.914276 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1341/2660] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:21.923054 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be793f5-7181-49fb-adef-0ae167cd6ff0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:21.923373 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be793f5-7181-49fb-adef-0ae167cd6ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:21.923783 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be793f5-7181-49fb-adef-0ae167cd6ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:21.945307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7be793f5-7181-49fb-adef-0ae167cd6ff0 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:21.946548 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7be793f5-7181-49fb-adef-0ae167cd6ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:21.955087 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1320/2661] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:21.959591 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9f20d-9c9f-4828-bc4d-9c5cfd2e1f63 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:21.970213 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9f20d-9c9f-4828-bc4d-9c5cfd2e1f63 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:21.975599 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d167f-d294-42bc-8ead-31d340e87af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:21.976210 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d167f-d294-42bc-8ead-31d340e87af9 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:21.976542 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d167f-d294-42bc-8ead-31d340e87af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:21.980378 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d167f-d294-42bc-8ead-31d340e87af9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:21.980378 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d167f-d294-42bc-8ead-31d340e87af9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:18:21.980378 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d167f-d294-42bc-8ead-31d340e87af9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:21.980378 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d167f-d294-42bc-8ead-31d340e87af9 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:21.992514 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1342/2662] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 13:18:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:18:21.996907 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be793f5-7181-49fb-adef-0ae167cd6ff0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:22.007192 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be793f5-7181-49fb-adef-0ae167cd6ff0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:22.011940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5a3d9-b09d-4459-8341-a16f30983967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:22.012467 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5a3d9-b09d-4459-8341-a16f30983967 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:22.012748 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5a3d9-b09d-4459-8341-a16f30983967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:22.013080 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5a3d9-b09d-4459-8341-a16f30983967 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:22.013777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5a3d9-b09d-4459-8341-a16f30983967 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:22.015901 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5a3d9-b09d-4459-8341-a16f30983967 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:22.139057 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1321/2663] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:18:21 2022] POST /identity/v3/projects => generated 409 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:18:22.144153 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d167f-d294-42bc-8ead-31d340e87af9 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:22.154597 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d167f-d294-42bc-8ead-31d340e87af9 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:22.163403 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:22.223554 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1c80f-050e-415f-89aa-524a38f30a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:22.223554 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1c80f-050e-415f-89aa-524a38f30a72 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:22.223554 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1c80f-050e-415f-89aa-524a38f30a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:22.224002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1c80f-050e-415f-89aa-524a38f30a72 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:22.224576 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1c80f-050e-415f-89aa-524a38f30a72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:22.226609 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1c80f-050e-415f-89aa-524a38f30a72 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:22.309123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1343/2664] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:18:22 2022] POST /identity/v3/users => generated 305 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:18:22.313201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5a3d9-b09d-4459-8341-a16f30983967 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:22.323925 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5a3d9-b09d-4459-8341-a16f30983967 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:22.328698 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a943c93-8c8b-45a4-bc3c-8080e658e7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:22.329290 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a943c93-8c8b-45a4-bc3c-8080e658e7a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:22.329622 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a943c93-8c8b-45a4-bc3c-8080e658e7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:22.329896 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a943c93-8c8b-45a4-bc3c-8080e658e7a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:22.330566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a943c93-8c8b-45a4-bc3c-8080e658e7a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:22.332194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a943c93-8c8b-45a4-bc3c-8080e658e7a1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:22.343641 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1322/2665] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:18:22.347358 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1c80f-050e-415f-89aa-524a38f30a72 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:22.359719 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1c80f-050e-415f-89aa-524a38f30a72 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:22.365171 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6c6e3-bae6-4b5e-815e-aad35708c6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:22.365777 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6c6e3-bae6-4b5e-815e-aad35708c6b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:22.366053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6c6e3-bae6-4b5e-815e-aad35708c6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:22.366324 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6c6e3-bae6-4b5e-815e-aad35708c6b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:22.367015 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c6c6e3-bae6-4b5e-815e-aad35708c6b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:22.368783 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c6c6e3-bae6-4b5e-815e-aad35708c6b0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:22.379896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1344/2666] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:18:22.384220 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a943c93-8c8b-45a4-bc3c-8080e658e7a1 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:22.394820 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a943c93-8c8b-45a4-bc3c-8080e658e7a1 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:22.399582 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189e50a-606b-4a0a-87c4-88f1d6c96d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:22.400238 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189e50a-606b-4a0a-87c4-88f1d6c96d83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:22.400518 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189e50a-606b-4a0a-87c4-88f1d6c96d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:22.400793 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189e50a-606b-4a0a-87c4-88f1d6c96d83 admin admin] PATH_INFO: `/v3/projects/88274064ee8e44b39fcfca5633ef3e9b/users/6861d390214a4a9fbaf49c83ee4ea7cd/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:22.403065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:22.416889 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189e50a-606b-4a0a-87c4-88f1d6c96d83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88274064ee8e44b39fcfca5633ef3e9b, user_id=6861d390214a4a9fbaf49c83ee4ea7cd, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:22.420315 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189e50a-606b-4a0a-87c4-88f1d6c96d83 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:22.532339 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1323/2667] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:18:22 2022] PUT /identity/v3/projects/88274064ee8e44b39fcfca5633ef3e9b/users/6861d390214a4a9fbaf49c83ee4ea7cd/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:18:22.542817 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95c2e0-a6ca-44bf-9d2b-0c9d59eb8184 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:22.543108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95c2e0-a6ca-44bf-9d2b-0c9d59eb8184 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:22.543436 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95c2e0-a6ca-44bf-9d2b-0c9d59eb8184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:22.572019 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-bb95c2e0-a6ca-44bf-9d2b-0c9d59eb8184 None None] MFA Rules not processed for user `6861d390214a4a9fbaf49c83ee4ea7cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:22.573276 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bb95c2e0-a6ca-44bf-9d2b-0c9d59eb8184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:22.639505 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1345/2668] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:22 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:22.644017 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189e50a-606b-4a0a-87c4-88f1d6c96d83 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:22.705328 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189e50a-606b-4a0a-87c4-88f1d6c96d83 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:22.710269 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2405f-5a3e-41ec-9220-ab28fb564bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:22.711036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2405f-5a3e-41ec-9220-ab28fb564bae admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:22.711371 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2405f-5a3e-41ec-9220-ab28fb564bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:22.711652 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2405f-5a3e-41ec-9220-ab28fb564bae admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:22.712265 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d2405f-5a3e-41ec-9220-ab28fb564bae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:22.714213 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d2405f-5a3e-41ec-9220-ab28fb564bae admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:22.789634 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1324/2669] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:18:22 2022] POST /identity/v3/projects => generated 407 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:18:22.793711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95c2e0-a6ca-44bf-9d2b-0c9d59eb8184 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:22.804599 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95c2e0-a6ca-44bf-9d2b-0c9d59eb8184 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:22.861500 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09b72f-7cbb-4c3a-9273-065125c415aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:22.862150 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09b72f-7cbb-4c3a-9273-065125c415aa admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:22.862436 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09b72f-7cbb-4c3a-9273-065125c415aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:22.862975 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09b72f-7cbb-4c3a-9273-065125c415aa admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:22.863629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf09b72f-7cbb-4c3a-9273-065125c415aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:22.865566 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf09b72f-7cbb-4c3a-9273-065125c415aa admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:22.942328 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1346/2670] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:18:22 2022] POST /identity/v3/users => generated 310 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:18:22.946357 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2405f-5a3e-41ec-9220-ab28fb564bae admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:22.957306 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2405f-5a3e-41ec-9220-ab28fb564bae admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:22.962038 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd5524-98ed-44ff-b8ea-327766660f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:22.962650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd5524-98ed-44ff-b8ea-327766660f6a admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:22.962938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd5524-98ed-44ff-b8ea-327766660f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:22.963239 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd5524-98ed-44ff-b8ea-327766660f6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:22.963880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd5524-98ed-44ff-b8ea-327766660f6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:22.965510 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd5524-98ed-44ff-b8ea-327766660f6a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:22.977880 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1325/2671] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:18:22.981540 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09b72f-7cbb-4c3a-9273-065125c415aa admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:22.993105 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09b72f-7cbb-4c3a-9273-065125c415aa admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:22.997834 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a006a2d-cdd4-4b34-8935-102e1d09436b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:22.998481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a006a2d-cdd4-4b34-8935-102e1d09436b admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:22.998758 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a006a2d-cdd4-4b34-8935-102e1d09436b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:22.999032 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a006a2d-cdd4-4b34-8935-102e1d09436b admin admin] PATH_INFO: `/v3/projects/8d266879c9a144e5a72053d28331a76b/users/86a13b07bb814f7d8823da5f4ad25c22/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:23.002774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:23.014364 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a006a2d-cdd4-4b34-8935-102e1d09436b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d266879c9a144e5a72053d28331a76b, user_id=86a13b07bb814f7d8823da5f4ad25c22, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:23.017025 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a006a2d-cdd4-4b34-8935-102e1d09436b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:23.069284 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1347/2672] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:18:22 2022] PUT /identity/v3/projects/8d266879c9a144e5a72053d28331a76b/users/86a13b07bb814f7d8823da5f4ad25c22/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:23.073287 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd5524-98ed-44ff-b8ea-327766660f6a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:23.083972 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd5524-98ed-44ff-b8ea-327766660f6a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:23.143881 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514ca94-f18d-4544-ad45-e55c26839801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:23.144494 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514ca94-f18d-4544-ad45-e55c26839801 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:23.144771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514ca94-f18d-4544-ad45-e55c26839801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:23.145046 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514ca94-f18d-4544-ad45-e55c26839801 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:23.145685 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5514ca94-f18d-4544-ad45-e55c26839801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:23.147293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5514ca94-f18d-4544-ad45-e55c26839801 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:23.158364 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1326/2673] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:18:23.162647 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a006a2d-cdd4-4b34-8935-102e1d09436b admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:23.173843 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a006a2d-cdd4-4b34-8935-102e1d09436b admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:23.178478 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e5fcb-7d8c-4dbe-a3c9-cc4b321b1f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:23.179122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e5fcb-7d8c-4dbe-a3c9-cc4b321b1f67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:23.179469 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e5fcb-7d8c-4dbe-a3c9-cc4b321b1f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:23.179744 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e5fcb-7d8c-4dbe-a3c9-cc4b321b1f67 admin admin] PATH_INFO: `/v3/projects/8d266879c9a144e5a72053d28331a76b/users/86a13b07bb814f7d8823da5f4ad25c22/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:23.181652 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8e5fcb-7d8c-4dbe-a3c9-cc4b321b1f67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d266879c9a144e5a72053d28331a76b, user_id=86a13b07bb814f7d8823da5f4ad25c22, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:23.184318 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8e5fcb-7d8c-4dbe-a3c9-cc4b321b1f67 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:23.194904 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1348/2674] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:18:23 2022] PUT /identity/v3/projects/8d266879c9a144e5a72053d28331a76b/users/86a13b07bb814f7d8823da5f4ad25c22/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:23.206309 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187929e-740c-422c-9a4a-5526892ab6ab None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:23.206601 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187929e-740c-422c-9a4a-5526892ab6ab None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:23.206916 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187929e-740c-422c-9a4a-5526892ab6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:23.235615 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9187929e-740c-422c-9a4a-5526892ab6ab None None] MFA Rules not processed for user `86a13b07bb814f7d8823da5f4ad25c22`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:23.236837 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9187929e-740c-422c-9a4a-5526892ab6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:23.289407 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:23.315650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1327/2675] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:23 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:23.326893 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e5fcb-7d8c-4dbe-a3c9-cc4b321b1f67 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:23.328649 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6c8e5fcb-7d8c-4dbe-a3c9-cc4b321b1f67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:23.385093 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e5fcb-7d8c-4dbe-a3c9-cc4b321b1f67 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:23.389441 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e8df5-0e9e-4b9d-9f79-bbcd41951487 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:23.390071 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e8df5-0e9e-4b9d-9f79-bbcd41951487 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:23.390351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e8df5-0e9e-4b9d-9f79-bbcd41951487 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:23.390623 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e8df5-0e9e-4b9d-9f79-bbcd41951487 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:23.399740 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866e8df5-0e9e-4b9d-9f79-bbcd41951487 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:23.401166 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866e8df5-0e9e-4b9d-9f79-bbcd41951487 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:23.411331 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1349/2676] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:23 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:23.759015 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9187929e-740c-422c-9a4a-5526892ab6ab None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:23.769876 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9187929e-740c-422c-9a4a-5526892ab6ab None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:23.773573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7cb7b-eff9-4255-97c5-c4d9658a0338 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:23.774300 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7cb7b-eff9-4255-97c5-c4d9658a0338 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:23.774588 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7cb7b-eff9-4255-97c5-c4d9658a0338 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:23.774905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7cb7b-eff9-4255-97c5-c4d9658a0338 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:23.832348 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff7cb7b-eff9-4255-97c5-c4d9658a0338 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:23.833890 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff7cb7b-eff9-4255-97c5-c4d9658a0338 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:23.842855 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1328/2677] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:23 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:26.243963 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26728b-c6f5-4f41-bd98-8f1d583201fc None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:26.244434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26728b-c6f5-4f41-bd98-8f1d583201fc None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:26.244521 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26728b-c6f5-4f41-bd98-8f1d583201fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:26.265311 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9c26728b-c6f5-4f41-bd98-8f1d583201fc None None] MFA Rules not processed for user `e42059e9af9842548ea08dcd39ff0b6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:26.266572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9c26728b-c6f5-4f41-bd98-8f1d583201fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:26.324989 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1350/2678] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 10 13:18:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:26.366017 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7cb7b-eff9-4255-97c5-c4d9658a0338 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:26.367499 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-eff7cb7b-eff9-4255-97c5-c4d9658a0338 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:26.442502 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7cb7b-eff9-4255-97c5-c4d9658a0338 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:26.447747 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992eb57-230e-47ae-8dee-59a0e0161c79 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59395bfa505c4540b3957deee78eb698', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:26.448362 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992eb57-230e-47ae-8dee-59a0e0161c79 service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:26.448674 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992eb57-230e-47ae-8dee-59a0e0161c79 service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:26.448985 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992eb57-230e-47ae-8dee-59a0e0161c79 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:26.469770 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d992eb57-230e-47ae-8dee-59a0e0161c79 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:26.471628 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d992eb57-230e-47ae-8dee-59a0e0161c79 service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:26.480402 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1329/2679] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 13:18:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:18:27.297314 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42ef8d-df0a-4019-b188-04ab30ebbe59 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:27.297780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42ef8d-df0a-4019-b188-04ab30ebbe59 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:27.297867 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42ef8d-df0a-4019-b188-04ab30ebbe59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:27.317584 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6c42ef8d-df0a-4019-b188-04ab30ebbe59 None None] MFA Rules not processed for user `e42059e9af9842548ea08dcd39ff0b6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:27.318805 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6c42ef8d-df0a-4019-b188-04ab30ebbe59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:27.326577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1351/2680] 10.222.0.49 () {58 vars in 967 bytes} [Wed Aug 10 13:18:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:27.339055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992eb57-230e-47ae-8dee-59a0e0161c79 service swift] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:27.350231 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992eb57-230e-47ae-8dee-59a0e0161c79 service swift] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:27.355164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd670e6b-ea02-4afa-8898-075dd6c4d5ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59395bfa505c4540b3957deee78eb698', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:27.355773 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd670e6b-ea02-4afa-8898-075dd6c4d5ac service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:27.356053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd670e6b-ea02-4afa-8898-075dd6c4d5ac service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:27.356326 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd670e6b-ea02-4afa-8898-075dd6c4d5ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:27.364912 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd670e6b-ea02-4afa-8898-075dd6c4d5ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:27.366465 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd670e6b-ea02-4afa-8898-075dd6c4d5ac service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:27.375344 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1330/2681] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 13:18:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:18:27.908393 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13447249-9ca6-428b-bc4b-b1af86202308 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:27.908780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13447249-9ca6-428b-bc4b-b1af86202308 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:27.909122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13447249-9ca6-428b-bc4b-b1af86202308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:27.929077 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-13447249-9ca6-428b-bc4b-b1af86202308 None None] MFA Rules not processed for user `e42059e9af9842548ea08dcd39ff0b6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:27.930294 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-13447249-9ca6-428b-bc4b-b1af86202308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:27.938361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1352/2682] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 10 13:18:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:27.962121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd670e6b-ea02-4afa-8898-075dd6c4d5ac service swift] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:27.962121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd670e6b-ea02-4afa-8898-075dd6c4d5ac service swift] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:27.966219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1998c3-5000-4fe6-92c1-15cc5b9b0b7e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59395bfa505c4540b3957deee78eb698', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:27.967524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1998c3-5000-4fe6-92c1-15cc5b9b0b7e service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:27.967808 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1998c3-5000-4fe6-92c1-15cc5b9b0b7e service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:27.968082 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1998c3-5000-4fe6-92c1-15cc5b9b0b7e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:27.976280 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1998c3-5000-4fe6-92c1-15cc5b9b0b7e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:27.977857 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1998c3-5000-4fe6-92c1-15cc5b9b0b7e service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:27.986163 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1331/2683] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 13:18:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:18:28.739481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdecf0e-0271-4bcc-9d57-c1d2b0701c73 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:28.739967 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdecf0e-0271-4bcc-9d57-c1d2b0701c73 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:28.740074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdecf0e-0271-4bcc-9d57-c1d2b0701c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:28.758753 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ccdecf0e-0271-4bcc-9d57-c1d2b0701c73 None None] MFA Rules not processed for user `e42059e9af9842548ea08dcd39ff0b6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:28.759986 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ccdecf0e-0271-4bcc-9d57-c1d2b0701c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:28.767769 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1353/2684] 10.222.0.49 () {58 vars in 967 bytes} [Wed Aug 10 13:18:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:28.780435 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1998c3-5000-4fe6-92c1-15cc5b9b0b7e service swift] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:28.790977 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1998c3-5000-4fe6-92c1-15cc5b9b0b7e service swift] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:28.796178 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6bd1a-d62b-4616-92cf-6f49b4ff5a6d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59395bfa505c4540b3957deee78eb698', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:28.796771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6bd1a-d62b-4616-92cf-6f49b4ff5a6d service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:28.797055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6bd1a-d62b-4616-92cf-6f49b4ff5a6d service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:28.797329 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6bd1a-d62b-4616-92cf-6f49b4ff5a6d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:28.805566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6bd1a-d62b-4616-92cf-6f49b4ff5a6d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:28.807132 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6bd1a-d62b-4616-92cf-6f49b4ff5a6d service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:28.815916 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1332/2685] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 13:18:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:18:31.422579 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdecf0e-0271-4bcc-9d57-c1d2b0701c73 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:31.424178 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ccdecf0e-0271-4bcc-9d57-c1d2b0701c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:31.492856 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdecf0e-0271-4bcc-9d57-c1d2b0701c73 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:31.499510 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f576906-844b-41f5-bf67-1c899744a5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:31.500290 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f576906-844b-41f5-bf67-1c899744a5a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:31.500576 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f576906-844b-41f5-bf67-1c899744a5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:31.500855 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f576906-844b-41f5-bf67-1c899744a5a8 admin admin] PATH_INFO: `/v3/users/41368bdb7a1648848f483eb4a12e3c38` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:31.502074 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f576906-844b-41f5-bf67-1c899744a5a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41368bdb7a1648848f483eb4a12e3c38)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:31.504035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f576906-844b-41f5-bf67-1c899744a5a8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:31.505757 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:31.641547 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5f576906-844b-41f5-bf67-1c899744a5a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41368bdb7a1648848f483eb4a12e3c38'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:31.648493 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5f576906-844b-41f5-bf67-1c899744a5a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41368bdb7a1648848f483eb4a12e3c38'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:31.702168 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5f576906-844b-41f5-bf67-1c899744a5a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41368bdb7a1648848f483eb4a12e3c38'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:31.702795 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5f576906-844b-41f5-bf67-1c899744a5a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41368bdb7a1648848f483eb4a12e3c38'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:31.703456 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5f576906-844b-41f5-bf67-1c899744a5a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41368bdb7a1648848f483eb4a12e3c38'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:31.722050 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1354/2686] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:18:31 2022] DELETE /identity/v3/users/41368bdb7a1648848f483eb4a12e3c38 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:31.726644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637097f-cea9-4680-9bed-bff124f4731d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:31.729586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637097f-cea9-4680-9bed-bff124f4731d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:31.729586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637097f-cea9-4680-9bed-bff124f4731d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:31.747682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1637097f-cea9-4680-9bed-bff124f4731d None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:31.749587 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1637097f-cea9-4680-9bed-bff124f4731d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:31.770381 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:31.809401 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1333/2687] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:31.819610 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f576906-844b-41f5-bf67-1c899744a5a8 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:31.821007 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5f576906-844b-41f5-bf67-1c899744a5a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:31.876833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f576906-844b-41f5-bf67-1c899744a5a8 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:31.880396 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa4931-4c99-4d34-b8c1-189b99ae0df9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:31.880988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa4931-4c99-4d34-b8c1-189b99ae0df9 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:31.881263 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa4931-4c99-4d34-b8c1-189b99ae0df9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:31.881531 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa4931-4c99-4d34-b8c1-189b99ae0df9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:31.891003 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aa4931-4c99-4d34-b8c1-189b99ae0df9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:31.892507 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aa4931-4c99-4d34-b8c1-189b99ae0df9 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:31.901559 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1355/2688] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:32.132500 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637097f-cea9-4680-9bed-bff124f4731d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:32.134033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1637097f-cea9-4680-9bed-bff124f4731d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:32.147021 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637097f-cea9-4680-9bed-bff124f4731d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:32.156919 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b18d6-af2b-4e26-99fb-48a19186c99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:32.158126 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b18d6-af2b-4e26-99fb-48a19186c99f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:32.158643 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b18d6-af2b-4e26-99fb-48a19186c99f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:32.158943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b18d6-af2b-4e26-99fb-48a19186c99f admin admin] PATH_INFO: `/v3/projects/2a1915ca912a42b88145ea1bf2dbc049` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:32.160641 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b18d6-af2b-4e26-99fb-48a19186c99f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a1915ca912a42b88145ea1bf2dbc049)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:32.163445 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b18d6-af2b-4e26-99fb-48a19186c99f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:32.276879 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-af2b18d6-af2b-4e26-99fb-48a19186c99f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a1915ca912a42b88145ea1bf2dbc049', 'request_id': 'req-af2b18d6-af2b-4e26-99fb-48a19186c99f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:32.283746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-af2b18d6-af2b-4e26-99fb-48a19186c99f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a1915ca912a42b88145ea1bf2dbc049', 'request_id': 'req-af2b18d6-af2b-4e26-99fb-48a19186c99f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:32.348232 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-af2b18d6-af2b-4e26-99fb-48a19186c99f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a1915ca912a42b88145ea1bf2dbc049', 'request_id': 'req-af2b18d6-af2b-4e26-99fb-48a19186c99f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:32.360124 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-af2b18d6-af2b-4e26-99fb-48a19186c99f admin admin] The token cache is being invalidate because project 2a1915ca912a42b88145ea1bf2dbc049 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:18:32.360525 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-af2b18d6-af2b-4e26-99fb-48a19186c99f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:32.361180 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-af2b18d6-af2b-4e26-99fb-48a19186c99f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:32.362943 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1334/2689] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:18:32 2022] DELETE /identity/v3/projects/2a1915ca912a42b88145ea1bf2dbc049 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:33.774898 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa4931-4c99-4d34-b8c1-189b99ae0df9 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:33.776452 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-53aa4931-4c99-4d34-b8c1-189b99ae0df9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:33.790009 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa4931-4c99-4d34-b8c1-189b99ae0df9 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:33.849461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1036d612-a47e-4f27-961c-b42981a85723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:33.850035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036d612-a47e-4f27-961c-b42981a85723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:33.850314 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036d612-a47e-4f27-961c-b42981a85723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:33.850584 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036d612-a47e-4f27-961c-b42981a85723 admin admin] PATH_INFO: `/v3/users/6861d390214a4a9fbaf49c83ee4ea7cd` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:33.852135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1036d612-a47e-4f27-961c-b42981a85723 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6861d390214a4a9fbaf49c83ee4ea7cd)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:33.854354 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1036d612-a47e-4f27-961c-b42981a85723 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:33.886439 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:34.035678 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1036d612-a47e-4f27-961c-b42981a85723 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6861d390214a4a9fbaf49c83ee4ea7cd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:34.042898 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1036d612-a47e-4f27-961c-b42981a85723 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6861d390214a4a9fbaf49c83ee4ea7cd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:34.142277 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1036d612-a47e-4f27-961c-b42981a85723 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6861d390214a4a9fbaf49c83ee4ea7cd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:34.142917 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1036d612-a47e-4f27-961c-b42981a85723 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6861d390214a4a9fbaf49c83ee4ea7cd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:34.143539 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1036d612-a47e-4f27-961c-b42981a85723 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6861d390214a4a9fbaf49c83ee4ea7cd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:34.160885 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1356/2690] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:18:33 2022] DELETE /identity/v3/users/6861d390214a4a9fbaf49c83ee4ea7cd => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:34.165499 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f40c3e-0f39-445b-b1cf-815001a7b65f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:34.165994 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f40c3e-0f39-445b-b1cf-815001a7b65f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:34.166086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f40c3e-0f39-445b-b1cf-815001a7b65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:34.185386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a3f40c3e-0f39-445b-b1cf-815001a7b65f None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:34.186587 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a3f40c3e-0f39-445b-b1cf-815001a7b65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:34.244415 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1335/2691] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:34.255110 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1036d612-a47e-4f27-961c-b42981a85723 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:34.257533 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1036d612-a47e-4f27-961c-b42981a85723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:34.313950 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1036d612-a47e-4f27-961c-b42981a85723 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:34.317597 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5235bfe-6d71-489c-bee5-c9550a8688e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:34.318201 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5235bfe-6d71-489c-bee5-c9550a8688e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:34.318488 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5235bfe-6d71-489c-bee5-c9550a8688e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:34.318754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5235bfe-6d71-489c-bee5-c9550a8688e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:34.328328 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5235bfe-6d71-489c-bee5-c9550a8688e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:34.329748 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5235bfe-6d71-489c-bee5-c9550a8688e5 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:34.339057 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1357/2692] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:34.746005 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f40c3e-0f39-445b-b1cf-815001a7b65f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:34.747658 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a3f40c3e-0f39-445b-b1cf-815001a7b65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:34.759651 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f40c3e-0f39-445b-b1cf-815001a7b65f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:34.764327 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b0c78-a53a-4928-b338-9ef60b9a8893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:34.764970 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b0c78-a53a-4928-b338-9ef60b9a8893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:34.765426 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b0c78-a53a-4928-b338-9ef60b9a8893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:34.765735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b0c78-a53a-4928-b338-9ef60b9a8893 admin admin] PATH_INFO: `/v3/projects/88274064ee8e44b39fcfca5633ef3e9b` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:34.766908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623b0c78-a53a-4928-b338-9ef60b9a8893 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88274064ee8e44b39fcfca5633ef3e9b)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:34.768793 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623b0c78-a53a-4928-b338-9ef60b9a8893 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:34.872087 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:34.875356 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-623b0c78-a53a-4928-b338-9ef60b9a8893 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88274064ee8e44b39fcfca5633ef3e9b', 'request_id': 'req-623b0c78-a53a-4928-b338-9ef60b9a8893'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:34.880954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-623b0c78-a53a-4928-b338-9ef60b9a8893 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88274064ee8e44b39fcfca5633ef3e9b', 'request_id': 'req-623b0c78-a53a-4928-b338-9ef60b9a8893'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:34.931331 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-623b0c78-a53a-4928-b338-9ef60b9a8893 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88274064ee8e44b39fcfca5633ef3e9b', 'request_id': 'req-623b0c78-a53a-4928-b338-9ef60b9a8893'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:34.945424 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-623b0c78-a53a-4928-b338-9ef60b9a8893 admin admin] The token cache is being invalidate because project 88274064ee8e44b39fcfca5633ef3e9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:18:34.945806 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-623b0c78-a53a-4928-b338-9ef60b9a8893 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:34.946500 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-623b0c78-a53a-4928-b338-9ef60b9a8893 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:34.949312 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1336/2693] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:18:34 2022] DELETE /identity/v3/projects/88274064ee8e44b39fcfca5633ef3e9b => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:34.952417 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5235bfe-6d71-489c-bee5-c9550a8688e5 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:34.962173 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5235bfe-6d71-489c-bee5-c9550a8688e5 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:35.027233 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e96f3f-979e-49c6-99e6-f77e4f6daf11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:35.027947 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e96f3f-979e-49c6-99e6-f77e4f6daf11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:35.028235 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e96f3f-979e-49c6-99e6-f77e4f6daf11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:35.028540 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e96f3f-979e-49c6-99e6-f77e4f6daf11 admin admin] PATH_INFO: `/v3/users/86a13b07bb814f7d8823da5f4ad25c22` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:35.029715 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e96f3f-979e-49c6-99e6-f77e4f6daf11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86a13b07bb814f7d8823da5f4ad25c22)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:35.033058 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e96f3f-979e-49c6-99e6-f77e4f6daf11 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:35.188732 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:35.190149 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-88e96f3f-979e-49c6-99e6-f77e4f6daf11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86a13b07bb814f7d8823da5f4ad25c22'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:35.196803 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-88e96f3f-979e-49c6-99e6-f77e4f6daf11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86a13b07bb814f7d8823da5f4ad25c22'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:35.308317 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-88e96f3f-979e-49c6-99e6-f77e4f6daf11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86a13b07bb814f7d8823da5f4ad25c22'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:35.308945 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-88e96f3f-979e-49c6-99e6-f77e4f6daf11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86a13b07bb814f7d8823da5f4ad25c22'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:35.309561 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-88e96f3f-979e-49c6-99e6-f77e4f6daf11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86a13b07bb814f7d8823da5f4ad25c22'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:35.329923 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1358/2694] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:18:34 2022] DELETE /identity/v3/users/86a13b07bb814f7d8823da5f4ad25c22 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:35.710199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b0c78-a53a-4928-b338-9ef60b9a8893 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:35.711681 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-623b0c78-a53a-4928-b338-9ef60b9a8893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:35.725580 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b0c78-a53a-4928-b338-9ef60b9a8893 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:35.784236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c83312-2f5e-46fd-b937-ec361718f191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:35.784874 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c83312-2f5e-46fd-b937-ec361718f191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:35.785291 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c83312-2f5e-46fd-b937-ec361718f191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:35.785568 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c83312-2f5e-46fd-b937-ec361718f191 admin admin] PATH_INFO: `/v3/projects/8d266879c9a144e5a72053d28331a76b` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:35.786746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c83312-2f5e-46fd-b937-ec361718f191 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d266879c9a144e5a72053d28331a76b)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:35.788720 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c83312-2f5e-46fd-b937-ec361718f191 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:35.976637 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-70c83312-2f5e-46fd-b937-ec361718f191 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d266879c9a144e5a72053d28331a76b', 'request_id': 'req-70c83312-2f5e-46fd-b937-ec361718f191'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:35.983312 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-70c83312-2f5e-46fd-b937-ec361718f191 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d266879c9a144e5a72053d28331a76b', 'request_id': 'req-70c83312-2f5e-46fd-b937-ec361718f191'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:36.028431 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-70c83312-2f5e-46fd-b937-ec361718f191 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d266879c9a144e5a72053d28331a76b', 'request_id': 'req-70c83312-2f5e-46fd-b937-ec361718f191'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:36.040219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-70c83312-2f5e-46fd-b937-ec361718f191 admin admin] The token cache is being invalidate because project 8d266879c9a144e5a72053d28331a76b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:18:36.040502 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-70c83312-2f5e-46fd-b937-ec361718f191 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:36.041179 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-70c83312-2f5e-46fd-b937-ec361718f191 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:36.042871 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1337/2695] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:18:35 2022] DELETE /identity/v3/projects/8d266879c9a144e5a72053d28331a76b => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:36.048666 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ade5b-0332-43b0-af5f-171ca156408f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:36.049180 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ade5b-0332-43b0-af5f-171ca156408f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:36.049271 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ade5b-0332-43b0-af5f-171ca156408f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:36.069312 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ad0ade5b-0332-43b0-af5f-171ca156408f None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:36.070502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ad0ade5b-0332-43b0-af5f-171ca156408f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:36.129183 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1359/2696] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:36.136870 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163be44-916d-4a87-9474-efa3f400bc03 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:36.137157 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163be44-916d-4a87-9474-efa3f400bc03 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:36.137488 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163be44-916d-4a87-9474-efa3f400bc03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:36.157425 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a163be44-916d-4a87-9474-efa3f400bc03 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:36.158642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a163be44-916d-4a87-9474-efa3f400bc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:36.166546 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1338/2697] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:36.170814 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ade5b-0332-43b0-af5f-171ca156408f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:36.181261 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ade5b-0332-43b0-af5f-171ca156408f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:36.185830 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157b400-3005-47b4-8d2b-36492b7aa679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:36.186431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157b400-3005-47b4-8d2b-36492b7aa679 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:36.186707 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157b400-3005-47b4-8d2b-36492b7aa679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:36.186978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157b400-3005-47b4-8d2b-36492b7aa679 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:36.188410 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c157b400-3005-47b4-8d2b-36492b7aa679 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:18:36.188721 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c157b400-3005-47b4-8d2b-36492b7aa679 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:36.190288 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c157b400-3005-47b4-8d2b-36492b7aa679 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:36.202229 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1360/2698] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 13:18:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:18:36.206496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163be44-916d-4a87-9474-efa3f400bc03 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:36.215609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163be44-916d-4a87-9474-efa3f400bc03 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:36.221086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299eeb44-8af2-4e9c-9127-4872b4ff0402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:36.221712 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299eeb44-8af2-4e9c-9127-4872b4ff0402 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:36.221957 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299eeb44-8af2-4e9c-9127-4872b4ff0402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:36.222227 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299eeb44-8af2-4e9c-9127-4872b4ff0402 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:36.222853 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299eeb44-8af2-4e9c-9127-4872b4ff0402 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:36.224808 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299eeb44-8af2-4e9c-9127-4872b4ff0402 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:36.304670 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1339/2699] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:18:36 2022] POST /identity/v3/projects => generated 365 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:18:36.308650 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157b400-3005-47b4-8d2b-36492b7aa679 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:36.319308 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157b400-3005-47b4-8d2b-36492b7aa679 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:36.373832 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f0056-6273-4679-8cff-e60929d84907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:36.374428 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f0056-6273-4679-8cff-e60929d84907 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:36.375244 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f0056-6273-4679-8cff-e60929d84907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:36.375516 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f0056-6273-4679-8cff-e60929d84907 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:36.376123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f0056-6273-4679-8cff-e60929d84907 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:36.378013 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f0056-6273-4679-8cff-e60929d84907 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:36.479402 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1361/2700] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:18:36 2022] POST /identity/v3/users => generated 283 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:18:36.483831 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299eeb44-8af2-4e9c-9127-4872b4ff0402 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:36.495018 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299eeb44-8af2-4e9c-9127-4872b4ff0402 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:36.500510 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9847d4a-ab29-485b-b9aa-e78abb9d782c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:36.501208 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9847d4a-ab29-485b-b9aa-e78abb9d782c admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:36.501487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9847d4a-ab29-485b-b9aa-e78abb9d782c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:36.501757 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9847d4a-ab29-485b-b9aa-e78abb9d782c admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:36.502400 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9847d4a-ab29-485b-b9aa-e78abb9d782c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:36.504022 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9847d4a-ab29-485b-b9aa-e78abb9d782c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:36.514003 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1340/2701] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:18:36.518540 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f0056-6273-4679-8cff-e60929d84907 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:36.528376 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f0056-6273-4679-8cff-e60929d84907 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:36.532926 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f99ab-0765-4255-b1e1-e91bbd2e0c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:36.533504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f99ab-0765-4255-b1e1-e91bbd2e0c37 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:36.533782 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f99ab-0765-4255-b1e1-e91bbd2e0c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:36.534050 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f99ab-0765-4255-b1e1-e91bbd2e0c37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:36.534672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f99ab-0765-4255-b1e1-e91bbd2e0c37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:36.536273 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f99ab-0765-4255-b1e1-e91bbd2e0c37 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:36.546709 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1362/2702] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:18:36.550574 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9847d4a-ab29-485b-b9aa-e78abb9d782c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:36.560569 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9847d4a-ab29-485b-b9aa-e78abb9d782c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:36.566030 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53e621-c74c-4e6e-85b7-509f67a4fc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:36.566677 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53e621-c74c-4e6e-85b7-509f67a4fc9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:36.566961 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53e621-c74c-4e6e-85b7-509f67a4fc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:36.567283 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53e621-c74c-4e6e-85b7-509f67a4fc9c admin admin] PATH_INFO: `/v3/projects/fb8273571cc24acb959450523181ae84/users/2dff0e3cdf754070a934a26edebeb7d4/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:36.582310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc53e621-c74c-4e6e-85b7-509f67a4fc9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb8273571cc24acb959450523181ae84, user_id=2dff0e3cdf754070a934a26edebeb7d4, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:36.584975 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc53e621-c74c-4e6e-85b7-509f67a4fc9c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:36.641377 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:36.645875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1341/2703] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:18:36 2022] PUT /identity/v3/projects/fb8273571cc24acb959450523181ae84/users/2dff0e3cdf754070a934a26edebeb7d4/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:36.658158 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77a330-07e0-427a-accc-ea4e868637ea None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:36.658432 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77a330-07e0-427a-accc-ea4e868637ea None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:36.658705 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77a330-07e0-427a-accc-ea4e868637ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:36.686551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ad77a330-07e0-427a-accc-ea4e868637ea None None] MFA Rules not processed for user `2dff0e3cdf754070a934a26edebeb7d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:36.688628 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ad77a330-07e0-427a-accc-ea4e868637ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:36.705982 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:36.756805 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1363/2704] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:18:36.761556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53e621-c74c-4e6e-85b7-509f67a4fc9c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:36.822977 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53e621-c74c-4e6e-85b7-509f67a4fc9c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:36.827734 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb7a25b-f654-472a-bbbe-9307c7e38412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:36.831141 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7a25b-f654-472a-bbbe-9307c7e38412 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:36.831141 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7a25b-f654-472a-bbbe-9307c7e38412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:36.831141 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7a25b-f654-472a-bbbe-9307c7e38412 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:36.831141 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb7a25b-f654-472a-bbbe-9307c7e38412 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:36.831487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb7a25b-f654-472a-bbbe-9307c7e38412 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:36.920313 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1342/2705] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:18:36 2022] POST /identity/v3/projects => generated 365 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:18:36.925179 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad77a330-07e0-427a-accc-ea4e868637ea None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:36.935823 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad77a330-07e0-427a-accc-ea4e868637ea None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:36.991832 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678441b8-3e32-40cc-8701-1481a675c87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:36.992433 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678441b8-3e32-40cc-8701-1481a675c87c admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:36.992712 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678441b8-3e32-40cc-8701-1481a675c87c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:36.992984 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678441b8-3e32-40cc-8701-1481a675c87c admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:36.993591 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678441b8-3e32-40cc-8701-1481a675c87c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:36.995684 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678441b8-3e32-40cc-8701-1481a675c87c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:37.085943 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1364/2706] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:18:36 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:18:37.089982 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb7a25b-f654-472a-bbbe-9307c7e38412 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:37.101040 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb7a25b-f654-472a-bbbe-9307c7e38412 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:37.105644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f765d-340a-49f2-8838-caadb2d91d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:37.106238 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f765d-340a-49f2-8838-caadb2d91d52 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:37.106516 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f765d-340a-49f2-8838-caadb2d91d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:37.106788 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f765d-340a-49f2-8838-caadb2d91d52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:37.107493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f765d-340a-49f2-8838-caadb2d91d52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:37.109735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f765d-340a-49f2-8838-caadb2d91d52 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:37.120448 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1343/2707] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:18:37.124731 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678441b8-3e32-40cc-8701-1481a675c87c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:37.134657 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678441b8-3e32-40cc-8701-1481a675c87c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:37.139288 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407e318-e113-43fb-af9b-cf258be74cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:37.139920 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407e318-e113-43fb-af9b-cf258be74cc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:37.140193 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407e318-e113-43fb-af9b-cf258be74cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:37.140462 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407e318-e113-43fb-af9b-cf258be74cc9 admin admin] PATH_INFO: `/v3/projects/8ebe50561c7449b1a4e14fdee14e0925/users/b83f156f6ece4ade954dbc5691b48bd5/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:37.155219 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407e318-e113-43fb-af9b-cf258be74cc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ebe50561c7449b1a4e14fdee14e0925, user_id=b83f156f6ece4ade954dbc5691b48bd5, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:37.158532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407e318-e113-43fb-af9b-cf258be74cc9 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:37.233598 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1365/2708] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:18:37 2022] PUT /identity/v3/projects/8ebe50561c7449b1a4e14fdee14e0925/users/b83f156f6ece4ade954dbc5691b48bd5/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:18:37.238473 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f765d-340a-49f2-8838-caadb2d91d52 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:37.248377 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f765d-340a-49f2-8838-caadb2d91d52 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:37.305886 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778105d3-8583-452b-a24d-cd5352a3da7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:37.306488 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778105d3-8583-452b-a24d-cd5352a3da7f admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:37.306766 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778105d3-8583-452b-a24d-cd5352a3da7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:37.307036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778105d3-8583-452b-a24d-cd5352a3da7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:37.307687 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778105d3-8583-452b-a24d-cd5352a3da7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:37.309261 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778105d3-8583-452b-a24d-cd5352a3da7f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:37.320274 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1344/2709] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:18:37.324530 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407e318-e113-43fb-af9b-cf258be74cc9 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:37.334929 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407e318-e113-43fb-af9b-cf258be74cc9 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:37.339721 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c63261-0ad3-469c-bd1a-ab6b71a480dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:37.340308 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c63261-0ad3-469c-bd1a-ab6b71a480dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:37.340587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c63261-0ad3-469c-bd1a-ab6b71a480dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:37.340863 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c63261-0ad3-469c-bd1a-ab6b71a480dd admin admin] PATH_INFO: `/v3/projects/8ebe50561c7449b1a4e14fdee14e0925/users/b83f156f6ece4ade954dbc5691b48bd5/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:37.342807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c63261-0ad3-469c-bd1a-ab6b71a480dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ebe50561c7449b1a4e14fdee14e0925, user_id=b83f156f6ece4ade954dbc5691b48bd5, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:37.346235 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c63261-0ad3-469c-bd1a-ab6b71a480dd admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:37.356240 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1366/2710] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:18:37 2022] PUT /identity/v3/projects/8ebe50561c7449b1a4e14fdee14e0925/users/b83f156f6ece4ade954dbc5691b48bd5/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:37.367212 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a754c9-9a73-4778-9f5b-bb99f14f86d3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:37.367489 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a754c9-9a73-4778-9f5b-bb99f14f86d3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:37.367763 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a754c9-9a73-4778-9f5b-bb99f14f86d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:37.396121 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-82a754c9-9a73-4778-9f5b-bb99f14f86d3 None None] MFA Rules not processed for user `b83f156f6ece4ade954dbc5691b48bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:37.397345 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-82a754c9-9a73-4778-9f5b-bb99f14f86d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:37.455373 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:37.463863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1345/2711] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:37 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:37.475822 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c63261-0ad3-469c-bd1a-ab6b71a480dd admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:37.477475 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d3c63261-0ad3-469c-bd1a-ab6b71a480dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:37.535800 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c63261-0ad3-469c-bd1a-ab6b71a480dd admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:37.539479 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34754ec-831f-4e44-845f-05883c1071e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:37.540081 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34754ec-831f-4e44-845f-05883c1071e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:37.540362 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34754ec-831f-4e44-845f-05883c1071e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:37.540641 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34754ec-831f-4e44-845f-05883c1071e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:37.549875 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34754ec-831f-4e44-845f-05883c1071e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:37.551337 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34754ec-831f-4e44-845f-05883c1071e3 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:37.560722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1367/2712] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:37 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:37.745289 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a754c9-9a73-4778-9f5b-bb99f14f86d3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:37.755934 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a754c9-9a73-4778-9f5b-bb99f14f86d3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:37.759496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c0c80-acab-4723-b5ef-e6c5fdef99e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:37.760094 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c0c80-acab-4723-b5ef-e6c5fdef99e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:37.760372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c0c80-acab-4723-b5ef-e6c5fdef99e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:37.760644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c0c80-acab-4723-b5ef-e6c5fdef99e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:37.817459 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c0c80-acab-4723-b5ef-e6c5fdef99e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:37.818882 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c0c80-acab-4723-b5ef-e6c5fdef99e9 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:37.828443 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1346/2713] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:43.957262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34754ec-831f-4e44-845f-05883c1071e3 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:43.971213 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:44.020122 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34754ec-831f-4e44-845f-05883c1071e3 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:44.024772 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841bc4b-9867-48d6-8ee9-951dadc49ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:44.025399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841bc4b-9867-48d6-8ee9-951dadc49ed0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:44.025682 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841bc4b-9867-48d6-8ee9-951dadc49ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:44.025989 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841bc4b-9867-48d6-8ee9-951dadc49ed0 admin admin] PATH_INFO: `/v3/users/2dff0e3cdf754070a934a26edebeb7d4` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:44.027085 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a841bc4b-9867-48d6-8ee9-951dadc49ed0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dff0e3cdf754070a934a26edebeb7d4)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:44.029032 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a841bc4b-9867-48d6-8ee9-951dadc49ed0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:44.218816 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a841bc4b-9867-48d6-8ee9-951dadc49ed0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dff0e3cdf754070a934a26edebeb7d4'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:44.225717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a841bc4b-9867-48d6-8ee9-951dadc49ed0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dff0e3cdf754070a934a26edebeb7d4'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:44.296690 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a841bc4b-9867-48d6-8ee9-951dadc49ed0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dff0e3cdf754070a934a26edebeb7d4'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:44.297341 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a841bc4b-9867-48d6-8ee9-951dadc49ed0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dff0e3cdf754070a934a26edebeb7d4'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:44.297949 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a841bc4b-9867-48d6-8ee9-951dadc49ed0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dff0e3cdf754070a934a26edebeb7d4'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:44.317366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1368/2714] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:18:43 2022] DELETE /identity/v3/users/2dff0e3cdf754070a934a26edebeb7d4 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:44.321833 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe60b13-67c9-47ca-a666-d527ca7b7e85 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:44.322271 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe60b13-67c9-47ca-a666-d527ca7b7e85 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:44.322396 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe60b13-67c9-47ca-a666-d527ca7b7e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:44.341087 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ebe60b13-67c9-47ca-a666-d527ca7b7e85 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:44.342303 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ebe60b13-67c9-47ca-a666-d527ca7b7e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:44.401158 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1347/2715] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:44.410922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841bc4b-9867-48d6-8ee9-951dadc49ed0 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:44.412678 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a841bc4b-9867-48d6-8ee9-951dadc49ed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:44.471739 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841bc4b-9867-48d6-8ee9-951dadc49ed0 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:44.475360 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948b87a-592f-4c3a-ab6e-b970fe274232 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:44.475967 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948b87a-592f-4c3a-ab6e-b970fe274232 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:44.476241 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948b87a-592f-4c3a-ab6e-b970fe274232 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:44.476511 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948b87a-592f-4c3a-ab6e-b970fe274232 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:44.485539 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948b87a-592f-4c3a-ab6e-b970fe274232 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:44.489556 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948b87a-592f-4c3a-ab6e-b970fe274232 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:44.497176 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1369/2716] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:44.910567 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe60b13-67c9-47ca-a666-d527ca7b7e85 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:44.912210 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ebe60b13-67c9-47ca-a666-d527ca7b7e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:44.927540 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe60b13-67c9-47ca-a666-d527ca7b7e85 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:44.932189 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c427d-91f2-41e4-beca-19c629bd410f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:44.936960 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c427d-91f2-41e4-beca-19c629bd410f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:44.936960 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c427d-91f2-41e4-beca-19c629bd410f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:44.936960 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c427d-91f2-41e4-beca-19c629bd410f admin admin] PATH_INFO: `/v3/projects/fb8273571cc24acb959450523181ae84` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:44.936960 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c427d-91f2-41e4-beca-19c629bd410f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb8273571cc24acb959450523181ae84)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:44.936960 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c427d-91f2-41e4-beca-19c629bd410f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:45.022990 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-419c427d-91f2-41e4-beca-19c629bd410f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb8273571cc24acb959450523181ae84', 'request_id': 'req-419c427d-91f2-41e4-beca-19c629bd410f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:45.028311 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-419c427d-91f2-41e4-beca-19c629bd410f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb8273571cc24acb959450523181ae84', 'request_id': 'req-419c427d-91f2-41e4-beca-19c629bd410f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:45.073627 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-419c427d-91f2-41e4-beca-19c629bd410f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb8273571cc24acb959450523181ae84', 'request_id': 'req-419c427d-91f2-41e4-beca-19c629bd410f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:45.085185 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-419c427d-91f2-41e4-beca-19c629bd410f admin admin] The token cache is being invalidate because project fb8273571cc24acb959450523181ae84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:18:45.085493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-419c427d-91f2-41e4-beca-19c629bd410f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:45.086096 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-419c427d-91f2-41e4-beca-19c629bd410f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:45.087591 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1348/2717] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:18:44 2022] DELETE /identity/v3/projects/fb8273571cc24acb959450523181ae84 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:45.091650 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948b87a-592f-4c3a-ab6e-b970fe274232 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:45.102086 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948b87a-592f-4c3a-ab6e-b970fe274232 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:45.134043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:45.157197 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699617c-805b-480f-954c-7e96f9945fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:45.158435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699617c-805b-480f-954c-7e96f9945fa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:45.158882 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699617c-805b-480f-954c-7e96f9945fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:45.159160 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699617c-805b-480f-954c-7e96f9945fa5 admin admin] PATH_INFO: `/v3/users/b83f156f6ece4ade954dbc5691b48bd5` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:45.160301 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6699617c-805b-480f-954c-7e96f9945fa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b83f156f6ece4ade954dbc5691b48bd5)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:45.162194 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6699617c-805b-480f-954c-7e96f9945fa5 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:45.392963 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6699617c-805b-480f-954c-7e96f9945fa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b83f156f6ece4ade954dbc5691b48bd5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:45.400017 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6699617c-805b-480f-954c-7e96f9945fa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b83f156f6ece4ade954dbc5691b48bd5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:45.525599 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6699617c-805b-480f-954c-7e96f9945fa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b83f156f6ece4ade954dbc5691b48bd5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:45.526263 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6699617c-805b-480f-954c-7e96f9945fa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b83f156f6ece4ade954dbc5691b48bd5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:45.526883 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6699617c-805b-480f-954c-7e96f9945fa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b83f156f6ece4ade954dbc5691b48bd5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:45.545604 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1370/2718] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:18:45 2022] DELETE /identity/v3/users/b83f156f6ece4ade954dbc5691b48bd5 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:45.949698 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c427d-91f2-41e4-beca-19c629bd410f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:45.951215 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-419c427d-91f2-41e4-beca-19c629bd410f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:45.964303 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c427d-91f2-41e4-beca-19c629bd410f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:45.979903 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:46.024214 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628fc06e-8718-4f61-acea-2eaa748d6d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:46.024850 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628fc06e-8718-4f61-acea-2eaa748d6d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:46.025127 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628fc06e-8718-4f61-acea-2eaa748d6d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:46.025401 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628fc06e-8718-4f61-acea-2eaa748d6d62 admin admin] PATH_INFO: `/v3/projects/8ebe50561c7449b1a4e14fdee14e0925` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:46.026508 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628fc06e-8718-4f61-acea-2eaa748d6d62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ebe50561c7449b1a4e14fdee14e0925)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:46.028480 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628fc06e-8718-4f61-acea-2eaa748d6d62 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:46.115245 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-628fc06e-8718-4f61-acea-2eaa748d6d62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ebe50561c7449b1a4e14fdee14e0925', 'request_id': 'req-628fc06e-8718-4f61-acea-2eaa748d6d62'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:46.120642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-628fc06e-8718-4f61-acea-2eaa748d6d62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ebe50561c7449b1a4e14fdee14e0925', 'request_id': 'req-628fc06e-8718-4f61-acea-2eaa748d6d62'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:46.165089 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-628fc06e-8718-4f61-acea-2eaa748d6d62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ebe50561c7449b1a4e14fdee14e0925', 'request_id': 'req-628fc06e-8718-4f61-acea-2eaa748d6d62'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:46.176123 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-628fc06e-8718-4f61-acea-2eaa748d6d62 admin admin] The token cache is being invalidate because project 8ebe50561c7449b1a4e14fdee14e0925 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:18:46.176414 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-628fc06e-8718-4f61-acea-2eaa748d6d62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:46.177044 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-628fc06e-8718-4f61-acea-2eaa748d6d62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:46.178511 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1349/2719] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:18:45 2022] DELETE /identity/v3/projects/8ebe50561c7449b1a4e14fdee14e0925 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:46.183587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ed8e5-0fd6-4852-be38-24934defe755 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:46.184061 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ed8e5-0fd6-4852-be38-24934defe755 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:46.184153 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ed8e5-0fd6-4852-be38-24934defe755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:46.205274 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-895ed8e5-0fd6-4852-be38-24934defe755 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:46.206596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-895ed8e5-0fd6-4852-be38-24934defe755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:46.264470 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1371/2720] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:46.273312 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c2f2b-24dc-4432-818b-3c4d47f85b70 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:46.273594 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c2f2b-24dc-4432-818b-3c4d47f85b70 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:46.273864 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c2f2b-24dc-4432-818b-3c4d47f85b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:46.293978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1b2c2f2b-24dc-4432-818b-3c4d47f85b70 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:46.295194 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1b2c2f2b-24dc-4432-818b-3c4d47f85b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:46.303805 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1350/2721] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:46.308078 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ed8e5-0fd6-4852-be38-24934defe755 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:46.318456 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ed8e5-0fd6-4852-be38-24934defe755 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:46.323088 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e05c3d-8be2-4bd3-b5a2-350fe3804c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:46.323709 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e05c3d-8be2-4bd3-b5a2-350fe3804c25 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:46.323986 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e05c3d-8be2-4bd3-b5a2-350fe3804c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:46.324259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e05c3d-8be2-4bd3-b5a2-350fe3804c25 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:46.324897 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e05c3d-8be2-4bd3-b5a2-350fe3804c25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:18:46.325197 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e05c3d-8be2-4bd3-b5a2-350fe3804c25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:46.326772 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e05c3d-8be2-4bd3-b5a2-350fe3804c25 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:46.338056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1372/2722] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 13:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:18:46.342349 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c2f2b-24dc-4432-818b-3c4d47f85b70 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:46.351522 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c2f2b-24dc-4432-818b-3c4d47f85b70 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:46.356151 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17d182-8cfa-4054-b081-79972bcbbf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:46.356735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17d182-8cfa-4054-b081-79972bcbbf1f admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:46.357010 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17d182-8cfa-4054-b081-79972bcbbf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:46.357285 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17d182-8cfa-4054-b081-79972bcbbf1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:46.357898 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17d182-8cfa-4054-b081-79972bcbbf1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:46.359813 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17d182-8cfa-4054-b081-79972bcbbf1f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:46.435067 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1351/2723] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:18:46 2022] POST /identity/v3/projects => generated 373 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:18:46.440164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e05c3d-8be2-4bd3-b5a2-350fe3804c25 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:46.450638 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e05c3d-8be2-4bd3-b5a2-350fe3804c25 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:46.490558 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:46.507265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496a241-1619-47ef-8d03-8a4d89e2d791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:46.507875 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496a241-1619-47ef-8d03-8a4d89e2d791 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:46.508156 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496a241-1619-47ef-8d03-8a4d89e2d791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:46.508430 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496a241-1619-47ef-8d03-8a4d89e2d791 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:46.509047 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2496a241-1619-47ef-8d03-8a4d89e2d791 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:46.510943 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2496a241-1619-47ef-8d03-8a4d89e2d791 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:46.622408 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1373/2724] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:18:46 2022] POST /identity/v3/users => generated 287 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:18:46.627054 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17d182-8cfa-4054-b081-79972bcbbf1f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:46.637502 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17d182-8cfa-4054-b081-79972bcbbf1f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:46.642202 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376af303-0482-4c39-bb63-db235152fef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:46.642976 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376af303-0482-4c39-bb63-db235152fef2 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:46.643300 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376af303-0482-4c39-bb63-db235152fef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:46.643564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376af303-0482-4c39-bb63-db235152fef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:46.644203 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376af303-0482-4c39-bb63-db235152fef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:46.645796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376af303-0482-4c39-bb63-db235152fef2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:46.656054 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1352/2725] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:18:46.660284 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496a241-1619-47ef-8d03-8a4d89e2d791 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:46.670310 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496a241-1619-47ef-8d03-8a4d89e2d791 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:46.675726 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f104d3a7-8b04-4599-8595-553e3d400a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:46.676315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104d3a7-8b04-4599-8595-553e3d400a10 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:46.676591 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104d3a7-8b04-4599-8595-553e3d400a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:46.676904 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104d3a7-8b04-4599-8595-553e3d400a10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:46.677531 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f104d3a7-8b04-4599-8595-553e3d400a10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:46.679088 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f104d3a7-8b04-4599-8595-553e3d400a10 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:46.690992 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1374/2726] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:18:46.694984 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376af303-0482-4c39-bb63-db235152fef2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:46.705354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376af303-0482-4c39-bb63-db235152fef2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:46.709905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc140e-2b69-4ba2-a130-b2f2dc425f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:46.710542 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc140e-2b69-4ba2-a130-b2f2dc425f2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:46.710817 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc140e-2b69-4ba2-a130-b2f2dc425f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:46.711089 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc140e-2b69-4ba2-a130-b2f2dc425f2f admin admin] PATH_INFO: `/v3/projects/1c67920f2eef4cdcab97eaa306db9449/users/658ca6778ebe4191a70afc973b89f8e7/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:46.727324 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc140e-2b69-4ba2-a130-b2f2dc425f2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c67920f2eef4cdcab97eaa306db9449, user_id=658ca6778ebe4191a70afc973b89f8e7, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:46.730058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc140e-2b69-4ba2-a130-b2f2dc425f2f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:46.806455 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1353/2727] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:18:46 2022] PUT /identity/v3/projects/1c67920f2eef4cdcab97eaa306db9449/users/658ca6778ebe4191a70afc973b89f8e7/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:46.818163 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184bfb3b-a5e3-42e8-9cbe-4300f60e8a85 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:46.818449 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184bfb3b-a5e3-42e8-9cbe-4300f60e8a85 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:46.818724 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184bfb3b-a5e3-42e8-9cbe-4300f60e8a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:46.846361 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-184bfb3b-a5e3-42e8-9cbe-4300f60e8a85 None None] MFA Rules not processed for user `658ca6778ebe4191a70afc973b89f8e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:46.847637 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-184bfb3b-a5e3-42e8-9cbe-4300f60e8a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:46.911516 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1375/2728] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:46.921977 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc140e-2b69-4ba2-a130-b2f2dc425f2f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:46.924855 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5dbc140e-2b69-4ba2-a130-b2f2dc425f2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:46.969940 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:46.980564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc140e-2b69-4ba2-a130-b2f2dc425f2f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:46.984280 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277274c1-d8fc-44fb-b891-43fa13c7809a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:46.984993 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277274c1-d8fc-44fb-b891-43fa13c7809a service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:46.985274 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277274c1-d8fc-44fb-b891-43fa13c7809a service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:46.985549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277274c1-d8fc-44fb-b891-43fa13c7809a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:46.994762 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277274c1-d8fc-44fb-b891-43fa13c7809a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:46.999363 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277274c1-d8fc-44fb-b891-43fa13c7809a service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:47.005873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1354/2729] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:50.882634 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184bfb3b-a5e3-42e8-9cbe-4300f60e8a85 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:50.944213 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184bfb3b-a5e3-42e8-9cbe-4300f60e8a85 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:50.948703 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd160a-5d31-4e62-89c7-c6273b1279cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:50.949329 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd160a-5d31-4e62-89c7-c6273b1279cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:50.949605 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd160a-5d31-4e62-89c7-c6273b1279cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:50.949876 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd160a-5d31-4e62-89c7-c6273b1279cf admin admin] PATH_INFO: `/v3/users/658ca6778ebe4191a70afc973b89f8e7` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:50.951008 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd160a-5d31-4e62-89c7-c6273b1279cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=658ca6778ebe4191a70afc973b89f8e7)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:50.952953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd160a-5d31-4e62-89c7-c6273b1279cf admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:51.103938 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2fbd160a-5d31-4e62-89c7-c6273b1279cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '658ca6778ebe4191a70afc973b89f8e7'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:51.108401 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:51.111237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2fbd160a-5d31-4e62-89c7-c6273b1279cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '658ca6778ebe4191a70afc973b89f8e7'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:51.211237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2fbd160a-5d31-4e62-89c7-c6273b1279cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '658ca6778ebe4191a70afc973b89f8e7'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:51.211808 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2fbd160a-5d31-4e62-89c7-c6273b1279cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '658ca6778ebe4191a70afc973b89f8e7'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:51.212443 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2fbd160a-5d31-4e62-89c7-c6273b1279cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '658ca6778ebe4191a70afc973b89f8e7'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:51.229664 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1376/2730] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:18:50 2022] DELETE /identity/v3/users/658ca6778ebe4191a70afc973b89f8e7 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:51.234237 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d2f0c-159e-4870-8282-9f86beceb03e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:51.234717 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d2f0c-159e-4870-8282-9f86beceb03e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:51.234866 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d2f0c-159e-4870-8282-9f86beceb03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:51.254814 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5b4d2f0c-159e-4870-8282-9f86beceb03e None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:51.256199 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5b4d2f0c-159e-4870-8282-9f86beceb03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:51.314100 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1355/2731] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:51.324537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd160a-5d31-4e62-89c7-c6273b1279cf admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:51.325933 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2fbd160a-5d31-4e62-89c7-c6273b1279cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:51.384367 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd160a-5d31-4e62-89c7-c6273b1279cf admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:51.388010 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2167c-e71e-4830-ad37-a3bf5007778a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:51.388631 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2167c-e71e-4830-ad37-a3bf5007778a service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:51.388905 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2167c-e71e-4830-ad37-a3bf5007778a service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:51.389174 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2167c-e71e-4830-ad37-a3bf5007778a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:51.398091 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2167c-e71e-4830-ad37-a3bf5007778a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:51.399538 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2167c-e71e-4830-ad37-a3bf5007778a service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:51.408669 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1377/2732] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:18:51.801610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d2f0c-159e-4870-8282-9f86beceb03e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:51.803088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5b4d2f0c-159e-4870-8282-9f86beceb03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:51.818959 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d2f0c-159e-4870-8282-9f86beceb03e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:51.824049 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48a4db-d576-48f9-bfe4-3207bd02840d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:51.824673 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48a4db-d576-48f9-bfe4-3207bd02840d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:51.825666 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48a4db-d576-48f9-bfe4-3207bd02840d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:51.825951 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48a4db-d576-48f9-bfe4-3207bd02840d admin admin] PATH_INFO: `/v3/projects/1c67920f2eef4cdcab97eaa306db9449` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:51.827328 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb48a4db-d576-48f9-bfe4-3207bd02840d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c67920f2eef4cdcab97eaa306db9449)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:51.829268 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb48a4db-d576-48f9-bfe4-3207bd02840d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:51.977848 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bb48a4db-d576-48f9-bfe4-3207bd02840d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c67920f2eef4cdcab97eaa306db9449', 'request_id': 'req-bb48a4db-d576-48f9-bfe4-3207bd02840d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:51.983390 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bb48a4db-d576-48f9-bfe4-3207bd02840d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c67920f2eef4cdcab97eaa306db9449', 'request_id': 'req-bb48a4db-d576-48f9-bfe4-3207bd02840d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:52.045081 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bb48a4db-d576-48f9-bfe4-3207bd02840d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c67920f2eef4cdcab97eaa306db9449', 'request_id': 'req-bb48a4db-d576-48f9-bfe4-3207bd02840d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:52.056492 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bb48a4db-d576-48f9-bfe4-3207bd02840d admin admin] The token cache is being invalidate because project 1c67920f2eef4cdcab97eaa306db9449 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:18:52.056785 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bb48a4db-d576-48f9-bfe4-3207bd02840d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:52.057436 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bb48a4db-d576-48f9-bfe4-3207bd02840d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:18:52.058916 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1356/2733] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:18:51 2022] DELETE /identity/v3/projects/1c67920f2eef4cdcab97eaa306db9449 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:52.063788 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50949f8e-f8d0-4506-9c21-172d0dd6ba9c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:52.064281 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50949f8e-f8d0-4506-9c21-172d0dd6ba9c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:52.064378 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50949f8e-f8d0-4506-9c21-172d0dd6ba9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:52.084034 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-50949f8e-f8d0-4506-9c21-172d0dd6ba9c None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:52.085237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-50949f8e-f8d0-4506-9c21-172d0dd6ba9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:52.143029 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1378/2734] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:52.151626 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc972603-2d2d-4c0c-8ca6-7c1b991d2c5f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:52.151919 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc972603-2d2d-4c0c-8ca6-7c1b991d2c5f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:52.152226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc972603-2d2d-4c0c-8ca6-7c1b991d2c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:52.172658 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-cc972603-2d2d-4c0c-8ca6-7c1b991d2c5f None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:52.173891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cc972603-2d2d-4c0c-8ca6-7c1b991d2c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:52.181844 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1357/2735] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:52.186108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50949f8e-f8d0-4506-9c21-172d0dd6ba9c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:52.195747 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50949f8e-f8d0-4506-9c21-172d0dd6ba9c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:52.200528 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc0f9d-0d46-49c3-8118-2b49e36d7615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:52.201153 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc0f9d-0d46-49c3-8118-2b49e36d7615 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:52.201435 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc0f9d-0d46-49c3-8118-2b49e36d7615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:52.201706 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc0f9d-0d46-49c3-8118-2b49e36d7615 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:52.202339 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc0f9d-0d46-49c3-8118-2b49e36d7615 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:18:52.202642 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc0f9d-0d46-49c3-8118-2b49e36d7615 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:52.204260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc0f9d-0d46-49c3-8118-2b49e36d7615 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:52.216151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1379/2736] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 13:18:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:18:52.220217 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc972603-2d2d-4c0c-8ca6-7c1b991d2c5f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:52.229862 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc972603-2d2d-4c0c-8ca6-7c1b991d2c5f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:52.234460 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e98b8-4a92-466d-ab64-3532a7d70e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:52.235042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e98b8-4a92-466d-ab64-3532a7d70e49 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:52.235344 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e98b8-4a92-466d-ab64-3532a7d70e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:52.235615 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e98b8-4a92-466d-ab64-3532a7d70e49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:52.236230 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e98b8-4a92-466d-ab64-3532a7d70e49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:52.238355 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e98b8-4a92-466d-ab64-3532a7d70e49 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:52.326892 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1358/2737] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:18:52 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:18:52.330973 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc0f9d-0d46-49c3-8118-2b49e36d7615 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:52.341799 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc0f9d-0d46-49c3-8118-2b49e36d7615 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:52.398120 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8470c4c-8cb1-4f2a-a85f-446f51cba6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:52.398736 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8470c4c-8cb1-4f2a-a85f-446f51cba6f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:52.398999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8470c4c-8cb1-4f2a-a85f-446f51cba6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:52.399299 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8470c4c-8cb1-4f2a-a85f-446f51cba6f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:52.399919 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8470c4c-8cb1-4f2a-a85f-446f51cba6f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:52.401864 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8470c4c-8cb1-4f2a-a85f-446f51cba6f9 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:52.409546 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:52.491258 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1380/2738] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:18:52 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:18:52.495420 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e98b8-4a92-466d-ab64-3532a7d70e49 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:52.506820 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e98b8-4a92-466d-ab64-3532a7d70e49 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:52.511452 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141306fb-5570-41f1-9a1e-0dd0a00db187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:52.512079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141306fb-5570-41f1-9a1e-0dd0a00db187 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:52.512356 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141306fb-5570-41f1-9a1e-0dd0a00db187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:52.512624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141306fb-5570-41f1-9a1e-0dd0a00db187 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:52.513248 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141306fb-5570-41f1-9a1e-0dd0a00db187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:52.514823 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141306fb-5570-41f1-9a1e-0dd0a00db187 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:52.525381 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1359/2739] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:18:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:18:52.529318 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8470c4c-8cb1-4f2a-a85f-446f51cba6f9 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:52.539575 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8470c4c-8cb1-4f2a-a85f-446f51cba6f9 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:52.544263 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7d96f-b033-4353-ae8d-2ca38e5af7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:52.544849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7d96f-b033-4353-ae8d-2ca38e5af7ba admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:52.545132 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7d96f-b033-4353-ae8d-2ca38e5af7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:52.545407 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7d96f-b033-4353-ae8d-2ca38e5af7ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:52.546044 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7d96f-b033-4353-ae8d-2ca38e5af7ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:52.547696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7d96f-b033-4353-ae8d-2ca38e5af7ba admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:52.558960 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1381/2740] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:18:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:18:52.562876 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141306fb-5570-41f1-9a1e-0dd0a00db187 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:52.573197 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141306fb-5570-41f1-9a1e-0dd0a00db187 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:52.577931 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915b3ed-7064-4d29-b760-a74937289763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:52.578499 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915b3ed-7064-4d29-b760-a74937289763 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:52.578776 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915b3ed-7064-4d29-b760-a74937289763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:52.579049 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915b3ed-7064-4d29-b760-a74937289763 admin admin] PATH_INFO: `/v3/projects/ca1183fd304e4160bfc22dc326a8b5a8/users/4f8477a2b3cc414b94b1bfaabf8e0e71/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:52.594981 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915b3ed-7064-4d29-b760-a74937289763 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca1183fd304e4160bfc22dc326a8b5a8, user_id=4f8477a2b3cc414b94b1bfaabf8e0e71, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:52.597730 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915b3ed-7064-4d29-b760-a74937289763 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:52.679871 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1360/2741] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:18:52 2022] PUT /identity/v3/projects/ca1183fd304e4160bfc22dc326a8b5a8/users/4f8477a2b3cc414b94b1bfaabf8e0e71/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:52.690325 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f5361-01b7-4ce1-9a26-701ea932d975 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:52.690605 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f5361-01b7-4ce1-9a26-701ea932d975 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:52.691597 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f5361-01b7-4ce1-9a26-701ea932d975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:52.718463 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e09f5361-01b7-4ce1-9a26-701ea932d975 None None] MFA Rules not processed for user `4f8477a2b3cc414b94b1bfaabf8e0e71`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:52.719746 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e09f5361-01b7-4ce1-9a26-701ea932d975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:52.784090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1382/2742] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:52.788971 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915b3ed-7064-4d29-b760-a74937289763 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:52.798078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:52.850829 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915b3ed-7064-4d29-b760-a74937289763 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:52.855564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2eb25f-5ae1-498d-aaf8-a13cf8c7d157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:52.856180 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2eb25f-5ae1-498d-aaf8-a13cf8c7d157 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:52.856658 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2eb25f-5ae1-498d-aaf8-a13cf8c7d157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:52.857104 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2eb25f-5ae1-498d-aaf8-a13cf8c7d157 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:52.858078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2eb25f-5ae1-498d-aaf8-a13cf8c7d157 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:52.860052 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2eb25f-5ae1-498d-aaf8-a13cf8c7d157 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:52.945106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1361/2743] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:18:52 2022] POST /identity/v3/projects => generated 383 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:18:52.951961 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f5361-01b7-4ce1-9a26-701ea932d975 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:52.963022 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f5361-01b7-4ce1-9a26-701ea932d975 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:53.018240 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815fd773-6edb-4fbd-b3ee-ae1088dbd6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:53.018844 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815fd773-6edb-4fbd-b3ee-ae1088dbd6c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:53.020205 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815fd773-6edb-4fbd-b3ee-ae1088dbd6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:53.020484 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815fd773-6edb-4fbd-b3ee-ae1088dbd6c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:53.021098 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815fd773-6edb-4fbd-b3ee-ae1088dbd6c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:53.023028 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815fd773-6edb-4fbd-b3ee-ae1088dbd6c7 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:53.108423 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1383/2744] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:18:52 2022] POST /identity/v3/users => generated 292 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:18:53.112339 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2eb25f-5ae1-498d-aaf8-a13cf8c7d157 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:53.123115 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2eb25f-5ae1-498d-aaf8-a13cf8c7d157 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:53.128072 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9189ea9d-39a6-4c8a-80f5-49bff9cb3763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:53.129233 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189ea9d-39a6-4c8a-80f5-49bff9cb3763 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:53.129619 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189ea9d-39a6-4c8a-80f5-49bff9cb3763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:53.129898 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189ea9d-39a6-4c8a-80f5-49bff9cb3763 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:53.130540 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9189ea9d-39a6-4c8a-80f5-49bff9cb3763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:53.132140 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9189ea9d-39a6-4c8a-80f5-49bff9cb3763 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:53.142664 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1362/2745] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:18:53.147139 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815fd773-6edb-4fbd-b3ee-ae1088dbd6c7 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:53.156959 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815fd773-6edb-4fbd-b3ee-ae1088dbd6c7 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:53.161610 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ee519-8347-4179-8569-d10f59b1b179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:53.162201 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ee519-8347-4179-8569-d10f59b1b179 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:53.162474 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ee519-8347-4179-8569-d10f59b1b179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:53.162746 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ee519-8347-4179-8569-d10f59b1b179 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:53.163404 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ee519-8347-4179-8569-d10f59b1b179 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:53.164969 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ee519-8347-4179-8569-d10f59b1b179 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:53.174698 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1384/2746] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:18:53.178686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9189ea9d-39a6-4c8a-80f5-49bff9cb3763 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:53.188605 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9189ea9d-39a6-4c8a-80f5-49bff9cb3763 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:53.193728 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b14f9d-6aa3-4d8f-b114-2912026aae71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:53.194366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b14f9d-6aa3-4d8f-b114-2912026aae71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:53.194960 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b14f9d-6aa3-4d8f-b114-2912026aae71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:53.194960 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b14f9d-6aa3-4d8f-b114-2912026aae71 admin admin] PATH_INFO: `/v3/projects/0404ce906e86452f9e7cf6b8cb666721/users/7a0c3ca7b1154981bdf3b6cecd6dd2b1/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:53.210211 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b14f9d-6aa3-4d8f-b114-2912026aae71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0404ce906e86452f9e7cf6b8cb666721, user_id=7a0c3ca7b1154981bdf3b6cecd6dd2b1, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:53.212891 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b14f9d-6aa3-4d8f-b114-2912026aae71 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:53.280403 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1363/2747] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:18:53 2022] PUT /identity/v3/projects/0404ce906e86452f9e7cf6b8cb666721/users/7a0c3ca7b1154981bdf3b6cecd6dd2b1/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:53.291774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fd22c-7bee-49b9-a1fd-4472b1f6a56f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:53.292060 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fd22c-7bee-49b9-a1fd-4472b1f6a56f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:53.292336 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fd22c-7bee-49b9-a1fd-4472b1f6a56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:53.320422 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-646fd22c-7bee-49b9-a1fd-4472b1f6a56f None None] MFA Rules not processed for user `7a0c3ca7b1154981bdf3b6cecd6dd2b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:53.321640 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-646fd22c-7bee-49b9-a1fd-4472b1f6a56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:53.356097 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:53.387840 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1385/2748] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:53.392222 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b14f9d-6aa3-4d8f-b114-2912026aae71 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:53.454086 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b14f9d-6aa3-4d8f-b114-2912026aae71 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:53.458829 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17fce9-6dfe-415b-b7f3-41716197af1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:53.459456 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17fce9-6dfe-415b-b7f3-41716197af1d admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:53.459743 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17fce9-6dfe-415b-b7f3-41716197af1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:53.460132 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17fce9-6dfe-415b-b7f3-41716197af1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:53.460754 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17fce9-6dfe-415b-b7f3-41716197af1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:53.462645 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17fce9-6dfe-415b-b7f3-41716197af1d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:53.567463 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1364/2749] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:18:53 2022] POST /identity/v3/projects => generated 385 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:18:53.571461 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fd22c-7bee-49b9-a1fd-4472b1f6a56f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:53.582694 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fd22c-7bee-49b9-a1fd-4472b1f6a56f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:53.637920 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7dac2-a0e1-46ab-a048-f91f8d6665d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:53.638452 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7dac2-a0e1-46ab-a048-f91f8d6665d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:53.638734 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7dac2-a0e1-46ab-a048-f91f8d6665d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:53.639011 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7dac2-a0e1-46ab-a048-f91f8d6665d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:53.639670 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae7dac2-a0e1-46ab-a048-f91f8d6665d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:53.641570 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae7dac2-a0e1-46ab-a048-f91f8d6665d0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:53.731021 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1386/2750] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:18:53 2022] POST /identity/v3/users => generated 299 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:18:53.735189 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17fce9-6dfe-415b-b7f3-41716197af1d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:53.746903 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17fce9-6dfe-415b-b7f3-41716197af1d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:53.751453 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70785f-b396-445b-b5fd-c8331c1acf5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:53.752041 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70785f-b396-445b-b5fd-c8331c1acf5d admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:53.752319 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70785f-b396-445b-b5fd-c8331c1acf5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:53.752590 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70785f-b396-445b-b5fd-c8331c1acf5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:53.753222 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70785f-b396-445b-b5fd-c8331c1acf5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:53.755942 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70785f-b396-445b-b5fd-c8331c1acf5d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:53.766316 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1365/2751] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 13:18:53.770490 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7dac2-a0e1-46ab-a048-f91f8d6665d0 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:53.780953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7dac2-a0e1-46ab-a048-f91f8d6665d0 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:53.786360 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64144e56-eaad-461a-883f-21a7fb1cc7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:53.787031 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64144e56-eaad-461a-883f-21a7fb1cc7e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:53.787477 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64144e56-eaad-461a-883f-21a7fb1cc7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:53.787754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64144e56-eaad-461a-883f-21a7fb1cc7e7 admin admin] PATH_INFO: `/v3/projects/e80414ef2f3b4a8faed8ff201fec7e3c/users/244ea00a00434468b87df2463e14eb94/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:53.803228 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64144e56-eaad-461a-883f-21a7fb1cc7e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e80414ef2f3b4a8faed8ff201fec7e3c, user_id=244ea00a00434468b87df2463e14eb94, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:53.805869 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64144e56-eaad-461a-883f-21a7fb1cc7e7 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:53.874058 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1387/2752] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:18:53 2022] PUT /identity/v3/projects/e80414ef2f3b4a8faed8ff201fec7e3c/users/244ea00a00434468b87df2463e14eb94/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:53.878398 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70785f-b396-445b-b5fd-c8331c1acf5d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:53.888557 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70785f-b396-445b-b5fd-c8331c1acf5d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:53.917443 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:53.946058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53769c7-42bd-4f6b-bf4a-47e2b52d9e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:53.946668 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53769c7-42bd-4f6b-bf4a-47e2b52d9e05 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:53.946979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53769c7-42bd-4f6b-bf4a-47e2b52d9e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:53.947351 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53769c7-42bd-4f6b-bf4a-47e2b52d9e05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:53.947990 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53769c7-42bd-4f6b-bf4a-47e2b52d9e05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:53.952612 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53769c7-42bd-4f6b-bf4a-47e2b52d9e05 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:53.960246 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1366/2753] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:18:53.964350 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64144e56-eaad-461a-883f-21a7fb1cc7e7 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:53.981999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64144e56-eaad-461a-883f-21a7fb1cc7e7 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:53.981999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5065c6d5-89d0-48e9-8601-54177e74b0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:53.981999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065c6d5-89d0-48e9-8601-54177e74b0b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:53.981999 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065c6d5-89d0-48e9-8601-54177e74b0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:53.987418 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065c6d5-89d0-48e9-8601-54177e74b0b8 admin admin] PATH_INFO: `/v3/projects/e80414ef2f3b4a8faed8ff201fec7e3c/users/244ea00a00434468b87df2463e14eb94/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:53.987418 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5065c6d5-89d0-48e9-8601-54177e74b0b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e80414ef2f3b4a8faed8ff201fec7e3c, user_id=244ea00a00434468b87df2463e14eb94, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:53.987418 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5065c6d5-89d0-48e9-8601-54177e74b0b8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:53.996674 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1388/2754] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:18:53 2022] PUT /identity/v3/projects/e80414ef2f3b4a8faed8ff201fec7e3c/users/244ea00a00434468b87df2463e14eb94/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:18:54.007133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff25e63-cf7b-4417-8fe1-73361997d61a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:54.007537 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff25e63-cf7b-4417-8fe1-73361997d61a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:54.007866 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff25e63-cf7b-4417-8fe1-73361997d61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:54.035607 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-cff25e63-cf7b-4417-8fe1-73361997d61a None None] MFA Rules not processed for user `244ea00a00434468b87df2463e14eb94`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:18:54.036920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cff25e63-cf7b-4417-8fe1-73361997d61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:54.103580 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1367/2755] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:18:54 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:18:54.115270 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5065c6d5-89d0-48e9-8601-54177e74b0b8 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:54.116965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5065c6d5-89d0-48e9-8601-54177e74b0b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:54.173799 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5065c6d5-89d0-48e9-8601-54177e74b0b8 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:54.177571 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a51f8-4925-4219-90ac-25608f45ca5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:54.178199 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a51f8-4925-4219-90ac-25608f45ca5d service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:54.178464 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a51f8-4925-4219-90ac-25608f45ca5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:54.178762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a51f8-4925-4219-90ac-25608f45ca5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:54.219264 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:18:54.234153 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a51f8-4925-4219-90ac-25608f45ca5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:54.235611 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a51f8-4925-4219-90ac-25608f45ca5d service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:54.244719 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1389/2756] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:56.199048 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff25e63-cf7b-4417-8fe1-73361997d61a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:56.200509 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cff25e63-cf7b-4417-8fe1-73361997d61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:18:56.212945 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff25e63-cf7b-4417-8fe1-73361997d61a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:56.216526 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96812d-9085-408c-8db5-214fa6654a78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:56.217122 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96812d-9085-408c-8db5-214fa6654a78 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:56.217404 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96812d-9085-408c-8db5-214fa6654a78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:56.217676 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96812d-9085-408c-8db5-214fa6654a78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:56.274823 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96812d-9085-408c-8db5-214fa6654a78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:56.276321 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96812d-9085-408c-8db5-214fa6654a78 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:56.285484 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1368/2757] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:18:57.491547 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a51f8-4925-4219-90ac-25608f45ca5d service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:18:57.500083 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a51f8-4925-4219-90ac-25608f45ca5d service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:18:57.504308 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086770f-2304-4e7e-b21e-075a9abf934c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:18:57.504924 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086770f-2304-4e7e-b21e-075a9abf934c service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:18:57.505205 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086770f-2304-4e7e-b21e-075a9abf934c service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:18:57.505479 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086770f-2304-4e7e-b21e-075a9abf934c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:18:57.514308 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5086770f-2304-4e7e-b21e-075a9abf934c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:18:57.515761 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5086770f-2304-4e7e-b21e-075a9abf934c service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:18:57.525153 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1390/2758] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:18:57 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:19:02.146075 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96812d-9085-408c-8db5-214fa6654a78 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:02.197255 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:02.210028 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96812d-9085-408c-8db5-214fa6654a78 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:02.214583 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3a5ad-3da3-459e-9a96-ec0cb9d81331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:02.215325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3a5ad-3da3-459e-9a96-ec0cb9d81331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:02.215624 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3a5ad-3da3-459e-9a96-ec0cb9d81331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:02.215920 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3a5ad-3da3-459e-9a96-ec0cb9d81331 admin admin] PATH_INFO: `/v3/users/4f8477a2b3cc414b94b1bfaabf8e0e71` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:02.217060 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e3a5ad-3da3-459e-9a96-ec0cb9d81331 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f8477a2b3cc414b94b1bfaabf8e0e71)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:02.218988 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e3a5ad-3da3-459e-9a96-ec0cb9d81331 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:02.409002 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e9e3a5ad-3da3-459e-9a96-ec0cb9d81331 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f8477a2b3cc414b94b1bfaabf8e0e71'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:02.456743 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e9e3a5ad-3da3-459e-9a96-ec0cb9d81331 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f8477a2b3cc414b94b1bfaabf8e0e71'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:02.457438 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e9e3a5ad-3da3-459e-9a96-ec0cb9d81331 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f8477a2b3cc414b94b1bfaabf8e0e71'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:02.458106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e9e3a5ad-3da3-459e-9a96-ec0cb9d81331 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f8477a2b3cc414b94b1bfaabf8e0e71'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:02.465421 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e9e3a5ad-3da3-459e-9a96-ec0cb9d81331 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f8477a2b3cc414b94b1bfaabf8e0e71'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:02.482830 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1369/2759] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:19:02 2022] DELETE /identity/v3/users/4f8477a2b3cc414b94b1bfaabf8e0e71 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:19:02.487447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50fad6-487f-434b-9a1a-06f9ec6f5a25 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:02.488828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50fad6-487f-434b-9a1a-06f9ec6f5a25 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:02.489010 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50fad6-487f-434b-9a1a-06f9ec6f5a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:02.509018 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-8c50fad6-487f-434b-9a1a-06f9ec6f5a25 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:19:02.510248 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8c50fad6-487f-434b-9a1a-06f9ec6f5a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:02.569112 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1391/2760] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:19:02.578784 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3a5ad-3da3-459e-9a96-ec0cb9d81331 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:02.580228 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e9e3a5ad-3da3-459e-9a96-ec0cb9d81331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:02.639818 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3a5ad-3da3-459e-9a96-ec0cb9d81331 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:02.643413 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5b0b7-a831-4637-84ec-731910af1e71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:02.644137 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5b0b7-a831-4637-84ec-731910af1e71 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:02.644524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5b0b7-a831-4637-84ec-731910af1e71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:02.644896 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5b0b7-a831-4637-84ec-731910af1e71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:02.653883 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5b0b7-a831-4637-84ec-731910af1e71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:02.655346 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5b0b7-a831-4637-84ec-731910af1e71 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:02.664750 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1370/2761] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:19:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:19:03.053747 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c50fad6-487f-434b-9a1a-06f9ec6f5a25 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:03.055426 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8c50fad6-487f-434b-9a1a-06f9ec6f5a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:03.067643 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c50fad6-487f-434b-9a1a-06f9ec6f5a25 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:03.072277 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3558b81-8321-40bd-9ba1-220f4eecce96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:03.072909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3558b81-8321-40bd-9ba1-220f4eecce96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:03.073186 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3558b81-8321-40bd-9ba1-220f4eecce96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:03.073460 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3558b81-8321-40bd-9ba1-220f4eecce96 admin admin] PATH_INFO: `/v3/projects/ca1183fd304e4160bfc22dc326a8b5a8` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:03.074575 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3558b81-8321-40bd-9ba1-220f4eecce96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca1183fd304e4160bfc22dc326a8b5a8)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:03.076613 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3558b81-8321-40bd-9ba1-220f4eecce96 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:03.184238 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f3558b81-8321-40bd-9ba1-220f4eecce96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca1183fd304e4160bfc22dc326a8b5a8', 'request_id': 'req-f3558b81-8321-40bd-9ba1-220f4eecce96'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:03.189857 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f3558b81-8321-40bd-9ba1-220f4eecce96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca1183fd304e4160bfc22dc326a8b5a8', 'request_id': 'req-f3558b81-8321-40bd-9ba1-220f4eecce96'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:03.245232 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f3558b81-8321-40bd-9ba1-220f4eecce96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca1183fd304e4160bfc22dc326a8b5a8', 'request_id': 'req-f3558b81-8321-40bd-9ba1-220f4eecce96'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:03.257190 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f3558b81-8321-40bd-9ba1-220f4eecce96 admin admin] The token cache is being invalidate because project ca1183fd304e4160bfc22dc326a8b5a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:19:03.257480 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f3558b81-8321-40bd-9ba1-220f4eecce96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:03.258148 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f3558b81-8321-40bd-9ba1-220f4eecce96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:03.259679 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1392/2762] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:19:03 2022] DELETE /identity/v3/projects/ca1183fd304e4160bfc22dc326a8b5a8 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:19:03.264128 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5b0b7-a831-4637-84ec-731910af1e71 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:03.274201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5b0b7-a831-4637-84ec-731910af1e71 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:03.330972 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf90a7-6728-41ed-985a-3a44bc93e001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:03.332966 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf90a7-6728-41ed-985a-3a44bc93e001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:03.333247 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf90a7-6728-41ed-985a-3a44bc93e001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:03.333518 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf90a7-6728-41ed-985a-3a44bc93e001 admin admin] PATH_INFO: `/v3/users/7a0c3ca7b1154981bdf3b6cecd6dd2b1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:03.334677 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daf90a7-6728-41ed-985a-3a44bc93e001 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a0c3ca7b1154981bdf3b6cecd6dd2b1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:03.336615 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daf90a7-6728-41ed-985a-3a44bc93e001 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:03.350362 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:03.448092 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1daf90a7-6728-41ed-985a-3a44bc93e001 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a0c3ca7b1154981bdf3b6cecd6dd2b1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:03.508513 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1daf90a7-6728-41ed-985a-3a44bc93e001 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a0c3ca7b1154981bdf3b6cecd6dd2b1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:03.509151 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1daf90a7-6728-41ed-985a-3a44bc93e001 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a0c3ca7b1154981bdf3b6cecd6dd2b1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:03.509766 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1daf90a7-6728-41ed-985a-3a44bc93e001 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a0c3ca7b1154981bdf3b6cecd6dd2b1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:03.517542 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1daf90a7-6728-41ed-985a-3a44bc93e001 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a0c3ca7b1154981bdf3b6cecd6dd2b1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:03.535134 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1371/2763] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:19:03 2022] DELETE /identity/v3/users/7a0c3ca7b1154981bdf3b6cecd6dd2b1 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:19:03.926819 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3558b81-8321-40bd-9ba1-220f4eecce96 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:03.929925 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f3558b81-8321-40bd-9ba1-220f4eecce96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:03.942128 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3558b81-8321-40bd-9ba1-220f4eecce96 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:03.992543 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:04.001145 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b918d948-6a26-4bf4-9185-ee3f69642ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:04.004521 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918d948-6a26-4bf4-9185-ee3f69642ba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:04.004521 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918d948-6a26-4bf4-9185-ee3f69642ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:04.004521 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918d948-6a26-4bf4-9185-ee3f69642ba9 admin admin] PATH_INFO: `/v3/projects/0404ce906e86452f9e7cf6b8cb666721` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:04.004521 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b918d948-6a26-4bf4-9185-ee3f69642ba9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0404ce906e86452f9e7cf6b8cb666721)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:04.005734 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b918d948-6a26-4bf4-9185-ee3f69642ba9 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:04.091300 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b918d948-6a26-4bf4-9185-ee3f69642ba9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0404ce906e86452f9e7cf6b8cb666721', 'request_id': 'req-b918d948-6a26-4bf4-9185-ee3f69642ba9'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:04.096734 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b918d948-6a26-4bf4-9185-ee3f69642ba9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0404ce906e86452f9e7cf6b8cb666721', 'request_id': 'req-b918d948-6a26-4bf4-9185-ee3f69642ba9'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:04.170478 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b918d948-6a26-4bf4-9185-ee3f69642ba9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0404ce906e86452f9e7cf6b8cb666721', 'request_id': 'req-b918d948-6a26-4bf4-9185-ee3f69642ba9'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:04.181898 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b918d948-6a26-4bf4-9185-ee3f69642ba9 admin admin] The token cache is being invalidate because project 0404ce906e86452f9e7cf6b8cb666721 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:19:04.182186 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b918d948-6a26-4bf4-9185-ee3f69642ba9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:04.182808 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b918d948-6a26-4bf4-9185-ee3f69642ba9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:04.184294 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1393/2764] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:19:03 2022] DELETE /identity/v3/projects/0404ce906e86452f9e7cf6b8cb666721 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:19:04.188304 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf90a7-6728-41ed-985a-3a44bc93e001 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:04.189751 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1daf90a7-6728-41ed-985a-3a44bc93e001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:04.202569 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf90a7-6728-41ed-985a-3a44bc93e001 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:04.258493 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479e9c4-e748-4185-a2c8-8768fb81bc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:04.259129 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479e9c4-e748-4185-a2c8-8768fb81bc91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:04.259442 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479e9c4-e748-4185-a2c8-8768fb81bc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:04.259719 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479e9c4-e748-4185-a2c8-8768fb81bc91 admin admin] PATH_INFO: `/v3/users/244ea00a00434468b87df2463e14eb94` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:04.260856 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9479e9c4-e748-4185-a2c8-8768fb81bc91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=244ea00a00434468b87df2463e14eb94)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:04.262788 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9479e9c4-e748-4185-a2c8-8768fb81bc91 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:04.430175 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9479e9c4-e748-4185-a2c8-8768fb81bc91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '244ea00a00434468b87df2463e14eb94'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:04.496809 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9479e9c4-e748-4185-a2c8-8768fb81bc91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '244ea00a00434468b87df2463e14eb94'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:04.497433 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9479e9c4-e748-4185-a2c8-8768fb81bc91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '244ea00a00434468b87df2463e14eb94'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:04.498175 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9479e9c4-e748-4185-a2c8-8768fb81bc91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '244ea00a00434468b87df2463e14eb94'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:04.506599 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9479e9c4-e748-4185-a2c8-8768fb81bc91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '244ea00a00434468b87df2463e14eb94'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:04.525602 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1372/2765] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:19:04 2022] DELETE /identity/v3/users/244ea00a00434468b87df2463e14eb94 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:19:04.921320 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b918d948-6a26-4bf4-9185-ee3f69642ba9 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:04.922871 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b918d948-6a26-4bf4-9185-ee3f69642ba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:04.937012 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b918d948-6a26-4bf4-9185-ee3f69642ba9 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:04.996424 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4333bd40-ca76-466c-ab60-67f646f6952d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:04.997064 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4333bd40-ca76-466c-ab60-67f646f6952d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:04.997343 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4333bd40-ca76-466c-ab60-67f646f6952d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:04.997621 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4333bd40-ca76-466c-ab60-67f646f6952d admin admin] PATH_INFO: `/v3/projects/e80414ef2f3b4a8faed8ff201fec7e3c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:04.998743 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4333bd40-ca76-466c-ab60-67f646f6952d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e80414ef2f3b4a8faed8ff201fec7e3c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:05.000706 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4333bd40-ca76-466c-ab60-67f646f6952d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:05.125915 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-4333bd40-ca76-466c-ab60-67f646f6952d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e80414ef2f3b4a8faed8ff201fec7e3c', 'request_id': 'req-4333bd40-ca76-466c-ab60-67f646f6952d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:05.132242 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-4333bd40-ca76-466c-ab60-67f646f6952d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e80414ef2f3b4a8faed8ff201fec7e3c', 'request_id': 'req-4333bd40-ca76-466c-ab60-67f646f6952d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:05.193621 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-4333bd40-ca76-466c-ab60-67f646f6952d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e80414ef2f3b4a8faed8ff201fec7e3c', 'request_id': 'req-4333bd40-ca76-466c-ab60-67f646f6952d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:05.205099 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-4333bd40-ca76-466c-ab60-67f646f6952d admin admin] The token cache is being invalidate because project e80414ef2f3b4a8faed8ff201fec7e3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:19:05.216086 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-4333bd40-ca76-466c-ab60-67f646f6952d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:05.216086 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-4333bd40-ca76-466c-ab60-67f646f6952d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:05.216086 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1394/2766] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:19:04 2022] DELETE /identity/v3/projects/e80414ef2f3b4a8faed8ff201fec7e3c => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:19:14.521839 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=74,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:15.201960 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=78,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:17.687023 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb507eef-71b2-475d-90cf-d14175609ad0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:17.687508 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb507eef-71b2-475d-90cf-d14175609ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:17.687595 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb507eef-71b2-475d-90cf-d14175609ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:17.707394 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-cb507eef-71b2-475d-90cf-d14175609ad0 None None] MFA Rules not processed for user `e42059e9af9842548ea08dcd39ff0b6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:19:17.708604 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cb507eef-71b2-475d-90cf-d14175609ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:17.766822 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1373/2767] 10.222.0.49 () {58 vars in 967 bytes} [Wed Aug 10 13:19:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:19:17.779418 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4333bd40-ca76-466c-ab60-67f646f6952d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:17.780821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4333bd40-ca76-466c-ab60-67f646f6952d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:17.850707 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4333bd40-ca76-466c-ab60-67f646f6952d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:17.855600 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1b620-03b3-4683-9be5-994ded4caf39 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59395bfa505c4540b3957deee78eb698', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:17.856197 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1b620-03b3-4683-9be5-994ded4caf39 service swift] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:17.856477 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1b620-03b3-4683-9be5-994ded4caf39 service swift] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:17.856751 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1b620-03b3-4683-9be5-994ded4caf39 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:17.865515 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1b620-03b3-4683-9be5-994ded4caf39 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:17.867083 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1b620-03b3-4683-9be5-994ded4caf39 service swift] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:17.874994 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1395/2768] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 13:19:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 13:19:25.852448 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb507eef-71b2-475d-90cf-d14175609ad0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:25.853985 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cb507eef-71b2-475d-90cf-d14175609ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:25.922964 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb507eef-71b2-475d-90cf-d14175609ad0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:25.927978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63dcf4-bee7-4ed8-9495-1966c63729fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:25.928599 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63dcf4-bee7-4ed8-9495-1966c63729fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:25.928873 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63dcf4-bee7-4ed8-9495-1966c63729fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:25.929144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63dcf4-bee7-4ed8-9495-1966c63729fb admin admin] PATH_INFO: `/v3/users/c025957730c04df5980279aaef8d0807` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:25.930294 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63dcf4-bee7-4ed8-9495-1966c63729fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c025957730c04df5980279aaef8d0807)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:25.932248 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63dcf4-bee7-4ed8-9495-1966c63729fb admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:26.125859 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7e63dcf4-bee7-4ed8-9495-1966c63729fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c025957730c04df5980279aaef8d0807'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:26.203769 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7e63dcf4-bee7-4ed8-9495-1966c63729fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c025957730c04df5980279aaef8d0807'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:26.204392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7e63dcf4-bee7-4ed8-9495-1966c63729fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c025957730c04df5980279aaef8d0807'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:26.204996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7e63dcf4-bee7-4ed8-9495-1966c63729fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c025957730c04df5980279aaef8d0807'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:26.212706 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7e63dcf4-bee7-4ed8-9495-1966c63729fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c025957730c04df5980279aaef8d0807'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:26.229556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1374/2769] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:19:25 2022] DELETE /identity/v3/users/c025957730c04df5980279aaef8d0807 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:19:26.484945 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1b620-03b3-4683-9be5-994ded4caf39 service swift] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:26.498738 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1b620-03b3-4683-9be5-994ded4caf39 service swift] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:26.557466 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b8848c-6fd1-4be5-92f5-2d7dcb7ebb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:26.558090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8848c-6fd1-4be5-92f5-2d7dcb7ebb58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:26.558367 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8848c-6fd1-4be5-92f5-2d7dcb7ebb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:26.558639 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8848c-6fd1-4be5-92f5-2d7dcb7ebb58 admin admin] PATH_INFO: `/v3/projects/bd9b653c3c794b7192e659f2557e7335` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:26.559824 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b8848c-6fd1-4be5-92f5-2d7dcb7ebb58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd9b653c3c794b7192e659f2557e7335)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:26.561696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b8848c-6fd1-4be5-92f5-2d7dcb7ebb58 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:26.649481 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e5b8848c-6fd1-4be5-92f5-2d7dcb7ebb58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd9b653c3c794b7192e659f2557e7335', 'request_id': 'req-e5b8848c-6fd1-4be5-92f5-2d7dcb7ebb58'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:26.655021 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e5b8848c-6fd1-4be5-92f5-2d7dcb7ebb58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd9b653c3c794b7192e659f2557e7335', 'request_id': 'req-e5b8848c-6fd1-4be5-92f5-2d7dcb7ebb58'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:26.706878 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e5b8848c-6fd1-4be5-92f5-2d7dcb7ebb58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd9b653c3c794b7192e659f2557e7335', 'request_id': 'req-e5b8848c-6fd1-4be5-92f5-2d7dcb7ebb58'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:26.719542 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e5b8848c-6fd1-4be5-92f5-2d7dcb7ebb58 admin admin] The token cache is being invalidate because project bd9b653c3c794b7192e659f2557e7335 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:19:26.719949 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e5b8848c-6fd1-4be5-92f5-2d7dcb7ebb58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:26.720495 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e5b8848c-6fd1-4be5-92f5-2d7dcb7ebb58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:26.722207 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1396/2770] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:19:26 2022] DELETE /identity/v3/projects/bd9b653c3c794b7192e659f2557e7335 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:19:26.726411 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63dcf4-bee7-4ed8-9495-1966c63729fb admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:26.729703 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7e63dcf4-bee7-4ed8-9495-1966c63729fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:26.758255 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63dcf4-bee7-4ed8-9495-1966c63729fb admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:26.781530 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:26.824973 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75279145-4f08-49c9-8411-cb8b47980018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:26.825615 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75279145-4f08-49c9-8411-cb8b47980018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:26.826804 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75279145-4f08-49c9-8411-cb8b47980018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:26.826804 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75279145-4f08-49c9-8411-cb8b47980018 admin admin] PATH_INFO: `/v3/users/955679b46cd942608b7e13d09410ea01` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:26.827371 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75279145-4f08-49c9-8411-cb8b47980018 admin admin] RBAC: Authorizing `identity:delete_user(user_id=955679b46cd942608b7e13d09410ea01)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:26.829328 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75279145-4f08-49c9-8411-cb8b47980018 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:27.021316 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-75279145-4f08-49c9-8411-cb8b47980018 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '955679b46cd942608b7e13d09410ea01'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:27.100854 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-75279145-4f08-49c9-8411-cb8b47980018 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '955679b46cd942608b7e13d09410ea01'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:27.101477 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-75279145-4f08-49c9-8411-cb8b47980018 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '955679b46cd942608b7e13d09410ea01'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:27.102109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-75279145-4f08-49c9-8411-cb8b47980018 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '955679b46cd942608b7e13d09410ea01'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:27.109425 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-75279145-4f08-49c9-8411-cb8b47980018 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '955679b46cd942608b7e13d09410ea01'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:27.127849 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1375/2771] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:19:26 2022] DELETE /identity/v3/users/955679b46cd942608b7e13d09410ea01 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:19:27.371513 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b8848c-6fd1-4be5-92f5-2d7dcb7ebb58 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:19:27.372983 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e5b8848c-6fd1-4be5-92f5-2d7dcb7ebb58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:19:27.386494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b8848c-6fd1-4be5-92f5-2d7dcb7ebb58 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:19:27.404038 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:27.443772 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf06a7-f247-4dcc-905c-83f1a7035071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:19:27.444405 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf06a7-f247-4dcc-905c-83f1a7035071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:19:27.444684 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf06a7-f247-4dcc-905c-83f1a7035071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:19:27.444957 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf06a7-f247-4dcc-905c-83f1a7035071 admin admin] PATH_INFO: `/v3/projects/622cbf961ce84556945747e21c173eca` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:19:27.446071 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baf06a7-f247-4dcc-905c-83f1a7035071 admin admin] RBAC: Authorizing `identity:delete_project(project_id=622cbf961ce84556945747e21c173eca)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:19:27.448001 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baf06a7-f247-4dcc-905c-83f1a7035071 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:19:27.542828 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2baf06a7-f247-4dcc-905c-83f1a7035071 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '622cbf961ce84556945747e21c173eca', 'request_id': 'req-2baf06a7-f247-4dcc-905c-83f1a7035071'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:27.548326 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2baf06a7-f247-4dcc-905c-83f1a7035071 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '622cbf961ce84556945747e21c173eca', 'request_id': 'req-2baf06a7-f247-4dcc-905c-83f1a7035071'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:27.609192 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2baf06a7-f247-4dcc-905c-83f1a7035071 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '622cbf961ce84556945747e21c173eca', 'request_id': 'req-2baf06a7-f247-4dcc-905c-83f1a7035071'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:27.621983 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2baf06a7-f247-4dcc-905c-83f1a7035071 admin admin] The token cache is being invalidate because project 622cbf961ce84556945747e21c173eca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:19:27.622270 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2baf06a7-f247-4dcc-905c-83f1a7035071 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:27.622906 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2baf06a7-f247-4dcc-905c-83f1a7035071 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:19:27.624497 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1397/2772] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:19:27 2022] DELETE /identity/v3/projects/622cbf961ce84556945747e21c173eca => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:19:37.124299 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=42,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:19:37.618222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:20:29.675120 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75279145-4f08-49c9-8411-cb8b47980018 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:20:29.676677 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-75279145-4f08-49c9-8411-cb8b47980018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:20:29.739365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75279145-4f08-49c9-8411-cb8b47980018 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:20:29.790350 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b85a71-d7ff-45fc-bf79-6b89b3b505ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:20:29.790992 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b85a71-d7ff-45fc-bf79-6b89b3b505ca service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:20:29.791330 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b85a71-d7ff-45fc-bf79-6b89b3b505ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:20:29.791604 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b85a71-d7ff-45fc-bf79-6b89b3b505ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:20:29.793333 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-73b85a71-d7ff-45fc-bf79-6b89b3b505ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:20:29.802506 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b85a71-d7ff-45fc-bf79-6b89b3b505ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:20:29.803950 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b85a71-d7ff-45fc-bf79-6b89b3b505ca service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:20:29.812127 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1376/2773] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:20:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:20:29.897055 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf06a7-f247-4dcc-905c-83f1a7035071 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:20:29.898506 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2baf06a7-f247-4dcc-905c-83f1a7035071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:20:29.910857 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf06a7-f247-4dcc-905c-83f1a7035071 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:20:29.914399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef810e-b09b-49ea-aada-95f77659391f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:20:29.914988 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef810e-b09b-49ea-aada-95f77659391f service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:20:29.915301 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef810e-b09b-49ea-aada-95f77659391f service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:20:29.915576 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef810e-b09b-49ea-aada-95f77659391f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:20:29.917243 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-caef810e-b09b-49ea-aada-95f77659391f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:20:29.999800 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caef810e-b09b-49ea-aada-95f77659391f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:20:30.001800 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caef810e-b09b-49ea-aada-95f77659391f service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:20:30.012918 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1398/2774] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:20:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:20:39.808508 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=46 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:20:40.007903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=39 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:21:06.742717 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b85a71-d7ff-45fc-bf79-6b89b3b505ca service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:21:06.744334 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-73b85a71-d7ff-45fc-bf79-6b89b3b505ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:21:06.866307 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b85a71-d7ff-45fc-bf79-6b89b3b505ca service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:21:06.871345 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef8195-b615-4159-831b-fb2411726584 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff152e22374407eb72caf648a763107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:21:06.871937 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef8195-b615-4159-831b-fb2411726584 service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:21:06.872213 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef8195-b615-4159-831b-fb2411726584 service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:21:06.872485 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef8195-b615-4159-831b-fb2411726584 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:21:06.874246 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-47ef8195-b615-4159-831b-fb2411726584 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:21:06.886026 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef8195-b615-4159-831b-fb2411726584 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:21:06.887618 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef8195-b615-4159-831b-fb2411726584 service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:21:06.898396 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1377/2775] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:21:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:21:16.894231 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=54 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:22:00.838764 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef810e-b09b-49ea-aada-95f77659391f service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:00.841799 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-caef810e-b09b-49ea-aada-95f77659391f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:00.890661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef810e-b09b-49ea-aada-95f77659391f service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:00.896444 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eaee0c-9be6-48ee-935f-f8d699357ab5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:00.897061 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eaee0c-9be6-48ee-935f-f8d699357ab5 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:00.903315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eaee0c-9be6-48ee-935f-f8d699357ab5 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:00.903315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eaee0c-9be6-48ee-935f-f8d699357ab5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:00.903315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c8eaee0c-9be6-48ee-935f-f8d699357ab5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:00.912459 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eaee0c-9be6-48ee-935f-f8d699357ab5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:00.914085 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eaee0c-9be6-48ee-935f-f8d699357ab5 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:00.925102 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1399/2776] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:22:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:22:01.021765 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef8195-b615-4159-831b-fb2411726584 service placement] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:01.023656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-47ef8195-b615-4159-831b-fb2411726584 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:01.036905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef8195-b615-4159-831b-fb2411726584 service placement] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:01.040326 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cadb11-b66b-4a22-9488-135d454cfe0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:01.040906 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cadb11-b66b-4a22-9488-135d454cfe0f service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:01.041183 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cadb11-b66b-4a22-9488-135d454cfe0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:01.041450 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cadb11-b66b-4a22-9488-135d454cfe0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:01.043233 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-39cadb11-b66b-4a22-9488-135d454cfe0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:01.067575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cadb11-b66b-4a22-9488-135d454cfe0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:01.069038 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cadb11-b66b-4a22-9488-135d454cfe0f service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:01.080644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1378/2777] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:22:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:22:04.966077 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eaee0c-9be6-48ee-935f-f8d699357ab5 service nova] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:04.976354 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eaee0c-9be6-48ee-935f-f8d699357ab5 service nova] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:04.979941 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56865620-7e74-4031-9b2d-6642739c2c13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:04.980565 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56865620-7e74-4031-9b2d-6642739c2c13 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:04.980849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56865620-7e74-4031-9b2d-6642739c2c13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:04.981123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56865620-7e74-4031-9b2d-6642739c2c13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:04.982849 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-56865620-7e74-4031-9b2d-6642739c2c13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:05.044604 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56865620-7e74-4031-9b2d-6642739c2c13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:05.046029 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56865620-7e74-4031-9b2d-6642739c2c13 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:05.056178 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1400/2778] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:22:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:22:08.519547 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cadb11-b66b-4a22-9488-135d454cfe0f service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:08.534091 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cadb11-b66b-4a22-9488-135d454cfe0f service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:08.539506 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf40f5-0d1c-4bbf-80f3-712ab072421a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff152e22374407eb72caf648a763107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:08.540201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf40f5-0d1c-4bbf-80f3-712ab072421a service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:08.540548 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf40f5-0d1c-4bbf-80f3-712ab072421a service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:08.540889 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf40f5-0d1c-4bbf-80f3-712ab072421a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:08.542817 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-58cf40f5-0d1c-4bbf-80f3-712ab072421a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:08.554956 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf40f5-0d1c-4bbf-80f3-712ab072421a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:08.556662 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf40f5-0d1c-4bbf-80f3-712ab072421a service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:08.568684 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1379/2779] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:22:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:22:10.556809 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56865620-7e74-4031-9b2d-6642739c2c13 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:10.558286 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-56865620-7e74-4031-9b2d-6642739c2c13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:10.651791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56865620-7e74-4031-9b2d-6642739c2c13 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:10.656235 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e057d5-e1b3-47b1-b394-dc537a512676 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:10.661345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e057d5-e1b3-47b1-b394-dc537a512676 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:10.661345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e057d5-e1b3-47b1-b394-dc537a512676 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:10.661345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e057d5-e1b3-47b1-b394-dc537a512676 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:10.661345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c4e057d5-e1b3-47b1-b394-dc537a512676 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:10.718489 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:22:10.749477 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e057d5-e1b3-47b1-b394-dc537a512676 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:10.750997 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e057d5-e1b3-47b1-b394-dc537a512676 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:10.770312 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1401/2780] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:22:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:22:12.304459 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf40f5-0d1c-4bbf-80f3-712ab072421a service placement] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:12.305966 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-58cf40f5-0d1c-4bbf-80f3-712ab072421a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:12.323207 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf40f5-0d1c-4bbf-80f3-712ab072421a service placement] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:12.328469 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5dbd66-ed23-4d09-ab99-78b2e9b47a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:12.329088 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5dbd66-ed23-4d09-ab99-78b2e9b47a76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:12.329371 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5dbd66-ed23-4d09-ab99-78b2e9b47a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:12.329648 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5dbd66-ed23-4d09-ab99-78b2e9b47a76 admin admin] PATH_INFO: `/v3/users/40c6361443a14e9ebd71855ba11cee10` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:12.330771 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5dbd66-ed23-4d09-ab99-78b2e9b47a76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40c6361443a14e9ebd71855ba11cee10)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:12.332726 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5dbd66-ed23-4d09-ab99-78b2e9b47a76 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:12.489303 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-aa5dbd66-ed23-4d09-ab99-78b2e9b47a76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40c6361443a14e9ebd71855ba11cee10'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:12.554933 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-aa5dbd66-ed23-4d09-ab99-78b2e9b47a76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40c6361443a14e9ebd71855ba11cee10'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:12.555610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-aa5dbd66-ed23-4d09-ab99-78b2e9b47a76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40c6361443a14e9ebd71855ba11cee10'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:12.556257 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-aa5dbd66-ed23-4d09-ab99-78b2e9b47a76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40c6361443a14e9ebd71855ba11cee10'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:12.564394 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-aa5dbd66-ed23-4d09-ab99-78b2e9b47a76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40c6361443a14e9ebd71855ba11cee10'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:12.582519 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1380/2781] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:22:12 2022] DELETE /identity/v3/users/40c6361443a14e9ebd71855ba11cee10 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:12.836295 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e057d5-e1b3-47b1-b394-dc537a512676 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:12.847425 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e057d5-e1b3-47b1-b394-dc537a512676 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:12.909900 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe97e5-48b4-44bc-a123-0743e1bddafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:12.910558 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe97e5-48b4-44bc-a123-0743e1bddafe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:12.910834 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe97e5-48b4-44bc-a123-0743e1bddafe admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:12.911107 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe97e5-48b4-44bc-a123-0743e1bddafe admin admin] PATH_INFO: `/v3/projects/0874b1e036b34f9188b4b5b003d59742` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:12.912254 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe97e5-48b4-44bc-a123-0743e1bddafe admin admin] RBAC: Authorizing `identity:delete_project(project_id=0874b1e036b34f9188b4b5b003d59742)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:12.914459 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe97e5-48b4-44bc-a123-0743e1bddafe admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:13.024833 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-cbbe97e5-48b4-44bc-a123-0743e1bddafe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0874b1e036b34f9188b4b5b003d59742', 'request_id': 'req-cbbe97e5-48b4-44bc-a123-0743e1bddafe'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:13.031159 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-cbbe97e5-48b4-44bc-a123-0743e1bddafe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0874b1e036b34f9188b4b5b003d59742', 'request_id': 'req-cbbe97e5-48b4-44bc-a123-0743e1bddafe'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:13.103483 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-cbbe97e5-48b4-44bc-a123-0743e1bddafe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0874b1e036b34f9188b4b5b003d59742', 'request_id': 'req-cbbe97e5-48b4-44bc-a123-0743e1bddafe'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:13.116031 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-cbbe97e5-48b4-44bc-a123-0743e1bddafe admin admin] The token cache is being invalidate because project 0874b1e036b34f9188b4b5b003d59742 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:22:13.116319 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-cbbe97e5-48b4-44bc-a123-0743e1bddafe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:13.116947 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-cbbe97e5-48b4-44bc-a123-0743e1bddafe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:13.118502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1402/2782] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:22:12 2022] DELETE /identity/v3/projects/0874b1e036b34f9188b4b5b003d59742 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:13.122566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5dbd66-ed23-4d09-ab99-78b2e9b47a76 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:13.124065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-aa5dbd66-ed23-4d09-ab99-78b2e9b47a76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:13.137374 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5dbd66-ed23-4d09-ab99-78b2e9b47a76 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:13.194117 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d0126-745c-4548-8d3d-242b7989ca37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:13.194738 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d0126-745c-4548-8d3d-242b7989ca37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:13.195018 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d0126-745c-4548-8d3d-242b7989ca37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:13.195334 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d0126-745c-4548-8d3d-242b7989ca37 admin admin] PATH_INFO: `/v3/users/f9fb27456558481fb89de506304d8606` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:13.196459 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451d0126-745c-4548-8d3d-242b7989ca37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9fb27456558481fb89de506304d8606)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:13.198362 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451d0126-745c-4548-8d3d-242b7989ca37 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:13.377980 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:22:13.380391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-451d0126-745c-4548-8d3d-242b7989ca37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9fb27456558481fb89de506304d8606'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:13.463364 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-451d0126-745c-4548-8d3d-242b7989ca37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9fb27456558481fb89de506304d8606'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:13.464013 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-451d0126-745c-4548-8d3d-242b7989ca37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9fb27456558481fb89de506304d8606'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:13.464620 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-451d0126-745c-4548-8d3d-242b7989ca37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9fb27456558481fb89de506304d8606'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:13.472848 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-451d0126-745c-4548-8d3d-242b7989ca37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9fb27456558481fb89de506304d8606'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:13.490542 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1381/2783] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:22:13 2022] DELETE /identity/v3/users/f9fb27456558481fb89de506304d8606 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:13.820650 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe97e5-48b4-44bc-a123-0743e1bddafe admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:13.822153 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cbbe97e5-48b4-44bc-a123-0743e1bddafe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:13.841900 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe97e5-48b4-44bc-a123-0743e1bddafe admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:13.882633 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:22:13.895056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ac451-2ff5-45e3-99db-d0e3250acf1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:13.895716 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ac451-2ff5-45e3-99db-d0e3250acf1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:13.895996 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ac451-2ff5-45e3-99db-d0e3250acf1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:13.896270 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ac451-2ff5-45e3-99db-d0e3250acf1e admin admin] PATH_INFO: `/v3/projects/50d85908bf494c6ea3ba23fd6291cb02` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:13.897389 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ac451-2ff5-45e3-99db-d0e3250acf1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=50d85908bf494c6ea3ba23fd6291cb02)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:13.899324 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ac451-2ff5-45e3-99db-d0e3250acf1e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:14.009459 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2f3ac451-2ff5-45e3-99db-d0e3250acf1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50d85908bf494c6ea3ba23fd6291cb02', 'request_id': 'req-2f3ac451-2ff5-45e3-99db-d0e3250acf1e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:14.016675 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2f3ac451-2ff5-45e3-99db-d0e3250acf1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50d85908bf494c6ea3ba23fd6291cb02', 'request_id': 'req-2f3ac451-2ff5-45e3-99db-d0e3250acf1e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:14.065006 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2f3ac451-2ff5-45e3-99db-d0e3250acf1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50d85908bf494c6ea3ba23fd6291cb02', 'request_id': 'req-2f3ac451-2ff5-45e3-99db-d0e3250acf1e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:14.077102 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2f3ac451-2ff5-45e3-99db-d0e3250acf1e admin admin] The token cache is being invalidate because project 50d85908bf494c6ea3ba23fd6291cb02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:22:14.078412 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2f3ac451-2ff5-45e3-99db-d0e3250acf1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:14.078778 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2f3ac451-2ff5-45e3-99db-d0e3250acf1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:14.080510 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1403/2784] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:22:13 2022] DELETE /identity/v3/projects/50d85908bf494c6ea3ba23fd6291cb02 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:14.094966 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d0126-745c-4548-8d3d-242b7989ca37 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:14.096597 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-451d0126-745c-4548-8d3d-242b7989ca37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:14.161324 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d0126-745c-4548-8d3d-242b7989ca37 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:14.215078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b11cb4-2899-461b-afd9-b785aa8e1cf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:14.215736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b11cb4-2899-461b-afd9-b785aa8e1cf2 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:14.216012 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b11cb4-2899-461b-afd9-b785aa8e1cf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:14.216283 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b11cb4-2899-461b-afd9-b785aa8e1cf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:14.218067 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-20b11cb4-2899-461b-afd9-b785aa8e1cf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:14.294325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b11cb4-2899-461b-afd9-b785aa8e1cf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:14.295842 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b11cb4-2899-461b-afd9-b785aa8e1cf2 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:14.307682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1382/2785] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:22:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:22:17.779816 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ac451-2ff5-45e3-99db-d0e3250acf1e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:17.781995 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2f3ac451-2ff5-45e3-99db-d0e3250acf1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:17.855925 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ac451-2ff5-45e3-99db-d0e3250acf1e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:17.861545 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe8113-de4c-4db6-a937-a9ea0f37b857 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff152e22374407eb72caf648a763107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:17.862637 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe8113-de4c-4db6-a937-a9ea0f37b857 service placement] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:17.862931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe8113-de4c-4db6-a937-a9ea0f37b857 service placement] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:17.863215 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe8113-de4c-4db6-a937-a9ea0f37b857 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:17.864978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8abe8113-de4c-4db6-a937-a9ea0f37b857 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:17.876903 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abe8113-de4c-4db6-a937-a9ea0f37b857 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:17.879087 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abe8113-de4c-4db6-a937-a9ea0f37b857 service placement] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:17.888728 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1404/2786] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:22:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:22:17.919416 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b11cb4-2899-461b-afd9-b785aa8e1cf2 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:17.920534 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-20b11cb4-2899-461b-afd9-b785aa8e1cf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:17.934439 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b11cb4-2899-461b-afd9-b785aa8e1cf2 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:17.939547 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784fa70-ce8a-4f52-b929-0c8954ab309b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:17.940184 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784fa70-ce8a-4f52-b929-0c8954ab309b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:17.940466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784fa70-ce8a-4f52-b929-0c8954ab309b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:17.940743 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784fa70-ce8a-4f52-b929-0c8954ab309b admin admin] PATH_INFO: `/v3/users/c0ada1accf1742b393e8a12c60074b9e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:17.941863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a784fa70-ce8a-4f52-b929-0c8954ab309b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0ada1accf1742b393e8a12c60074b9e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:17.943848 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a784fa70-ce8a-4f52-b929-0c8954ab309b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:17.967293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:22:18.122541 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a784fa70-ce8a-4f52-b929-0c8954ab309b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0ada1accf1742b393e8a12c60074b9e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:18.183340 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a784fa70-ce8a-4f52-b929-0c8954ab309b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0ada1accf1742b393e8a12c60074b9e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:18.183959 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a784fa70-ce8a-4f52-b929-0c8954ab309b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0ada1accf1742b393e8a12c60074b9e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:18.184565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a784fa70-ce8a-4f52-b929-0c8954ab309b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0ada1accf1742b393e8a12c60074b9e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:18.191838 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a784fa70-ce8a-4f52-b929-0c8954ab309b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0ada1accf1742b393e8a12c60074b9e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:18.209361 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1383/2787] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:22:17 2022] DELETE /identity/v3/users/c0ada1accf1742b393e8a12c60074b9e => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:18.464877 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe8113-de4c-4db6-a937-a9ea0f37b857 service placement] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:18.479701 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe8113-de4c-4db6-a937-a9ea0f37b857 service placement] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:18.540536 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36f78d-12fa-45e3-a925-91bff67669f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:18.541166 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36f78d-12fa-45e3-a925-91bff67669f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:18.541445 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36f78d-12fa-45e3-a925-91bff67669f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:18.541730 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36f78d-12fa-45e3-a925-91bff67669f2 admin admin] PATH_INFO: `/v3/projects/9a40b36dd9944cb88fd54181f7f29685` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:18.542836 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36f78d-12fa-45e3-a925-91bff67669f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a40b36dd9944cb88fd54181f7f29685)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:18.544809 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36f78d-12fa-45e3-a925-91bff67669f2 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:18.555442 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:22:18.652279 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6b36f78d-12fa-45e3-a925-91bff67669f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a40b36dd9944cb88fd54181f7f29685', 'request_id': 'req-6b36f78d-12fa-45e3-a925-91bff67669f2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:18.659023 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6b36f78d-12fa-45e3-a925-91bff67669f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a40b36dd9944cb88fd54181f7f29685', 'request_id': 'req-6b36f78d-12fa-45e3-a925-91bff67669f2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:18.742838 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6b36f78d-12fa-45e3-a925-91bff67669f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a40b36dd9944cb88fd54181f7f29685', 'request_id': 'req-6b36f78d-12fa-45e3-a925-91bff67669f2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:18.754495 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6b36f78d-12fa-45e3-a925-91bff67669f2 admin admin] The token cache is being invalidate because project 9a40b36dd9944cb88fd54181f7f29685 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:22:18.754782 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6b36f78d-12fa-45e3-a925-91bff67669f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:18.755457 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6b36f78d-12fa-45e3-a925-91bff67669f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:18.756967 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1405/2788] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:22:18 2022] DELETE /identity/v3/projects/9a40b36dd9944cb88fd54181f7f29685 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:18.762979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b300f-393a-4aeb-8e96-153668d977e9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:18.763292 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b300f-393a-4aeb-8e96-153668d977e9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:18.763566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b300f-393a-4aeb-8e96-153668d977e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:18.784041 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5f9b300f-393a-4aeb-8e96-153668d977e9 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:22:18.785939 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5f9b300f-393a-4aeb-8e96-153668d977e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:18.842984 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1384/2789] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:22:18.853072 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73878081-1c55-4761-a66f-dc94fd7034b5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:18.853554 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73878081-1c55-4761-a66f-dc94fd7034b5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:18.853649 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73878081-1c55-4761-a66f-dc94fd7034b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:18.873446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-73878081-1c55-4761-a66f-dc94fd7034b5 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:22:18.875246 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-73878081-1c55-4761-a66f-dc94fd7034b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:18.887698 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1406/2790] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:22:18.889085 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b300f-393a-4aeb-8e96-153668d977e9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:18.899905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b300f-393a-4aeb-8e96-153668d977e9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:18.904593 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7b920-22a5-410e-9dd9-09ce0ab2d02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:18.905315 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7b920-22a5-410e-9dd9-09ce0ab2d02e admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:18.905608 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7b920-22a5-410e-9dd9-09ce0ab2d02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:18.906777 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7b920-22a5-410e-9dd9-09ce0ab2d02e admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:18.907552 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7b920-22a5-410e-9dd9-09ce0ab2d02e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:22:18.912532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7b920-22a5-410e-9dd9-09ce0ab2d02e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:18.912532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7b920-22a5-410e-9dd9-09ce0ab2d02e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:18.925366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1385/2791] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 13:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:22:18.926444 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73878081-1c55-4761-a66f-dc94fd7034b5 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:18.936200 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73878081-1c55-4761-a66f-dc94fd7034b5 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:18.941273 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc6241-5a94-4e2b-9144-035a85af02a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:18.941712 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc6241-5a94-4e2b-9144-035a85af02a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:18.941916 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc6241-5a94-4e2b-9144-035a85af02a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:18.942183 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc6241-5a94-4e2b-9144-035a85af02a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:18.942815 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc6241-5a94-4e2b-9144-035a85af02a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:18.944757 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc6241-5a94-4e2b-9144-035a85af02a8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:19.030922 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1407/2792] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:22:18 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:22:19.035431 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7b920-22a5-410e-9dd9-09ce0ab2d02e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:19.045308 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7b920-22a5-410e-9dd9-09ce0ab2d02e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:19.104569 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3daa66-cc28-463d-98f8-132502e5d388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:19.105174 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3daa66-cc28-463d-98f8-132502e5d388 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:19.105455 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3daa66-cc28-463d-98f8-132502e5d388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:19.105737 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3daa66-cc28-463d-98f8-132502e5d388 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:19.106362 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3daa66-cc28-463d-98f8-132502e5d388 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:19.108432 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3daa66-cc28-463d-98f8-132502e5d388 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:19.194324 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1386/2793] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:22:19 2022] POST /identity/v3/users => generated 287 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:22:19.198632 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc6241-5a94-4e2b-9144-035a85af02a8 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:19.209403 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc6241-5a94-4e2b-9144-035a85af02a8 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:19.214262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b431a3-627c-4a21-b9ca-4aace59b0a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:19.214893 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b431a3-627c-4a21-b9ca-4aace59b0a3e admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:19.215189 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b431a3-627c-4a21-b9ca-4aace59b0a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:19.215474 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b431a3-627c-4a21-b9ca-4aace59b0a3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:19.216234 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b431a3-627c-4a21-b9ca-4aace59b0a3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:19.217894 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b431a3-627c-4a21-b9ca-4aace59b0a3e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:19.229186 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1408/2794] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:22:19.233396 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3daa66-cc28-463d-98f8-132502e5d388 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:19.244338 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3daa66-cc28-463d-98f8-132502e5d388 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:19.249081 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2f9aa-a16c-477b-9d6c-37e96cb1b543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:19.249804 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2f9aa-a16c-477b-9d6c-37e96cb1b543 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:19.250090 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2f9aa-a16c-477b-9d6c-37e96cb1b543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:19.250365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2f9aa-a16c-477b-9d6c-37e96cb1b543 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:19.251015 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2f9aa-a16c-477b-9d6c-37e96cb1b543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:19.252738 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2f9aa-a16c-477b-9d6c-37e96cb1b543 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:19.264274 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1387/2795] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:22:19.268892 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b431a3-627c-4a21-b9ca-4aace59b0a3e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:19.281779 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b431a3-627c-4a21-b9ca-4aace59b0a3e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:19.285840 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e263974-628a-4a19-9aa8-ff8813b98d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:19.286487 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e263974-628a-4a19-9aa8-ff8813b98d99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:19.286770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e263974-628a-4a19-9aa8-ff8813b98d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:19.287046 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e263974-628a-4a19-9aa8-ff8813b98d99 admin admin] PATH_INFO: `/v3/projects/97f38fc27a764f499abdd2133d0c4a4b/users/5d17c654bade41c58d2c34372bee5adf/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:19.302222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e263974-628a-4a19-9aa8-ff8813b98d99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97f38fc27a764f499abdd2133d0c4a4b, user_id=5d17c654bade41c58d2c34372bee5adf, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:19.304913 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e263974-628a-4a19-9aa8-ff8813b98d99 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:19.372901 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1409/2796] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:22:19 2022] PUT /identity/v3/projects/97f38fc27a764f499abdd2133d0c4a4b/users/5d17c654bade41c58d2c34372bee5adf/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:19.385611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed856c-5b80-46ba-b49f-dbb62179a1ea None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:19.385903 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed856c-5b80-46ba-b49f-dbb62179a1ea None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:19.386189 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed856c-5b80-46ba-b49f-dbb62179a1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:19.395703 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:22:19.421604 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-94ed856c-5b80-46ba-b49f-dbb62179a1ea None None] MFA Rules not processed for user `5d17c654bade41c58d2c34372bee5adf`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:22:19.422838 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-94ed856c-5b80-46ba-b49f-dbb62179a1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:19.495812 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1388/2797] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:22:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:22:19.508035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e263974-628a-4a19-9aa8-ff8813b98d99 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:19.509905 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1e263974-628a-4a19-9aa8-ff8813b98d99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:19.594976 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e263974-628a-4a19-9aa8-ff8813b98d99 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:19.601430 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce3857-09bc-444a-b498-78d78042039b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:19.602115 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce3857-09bc-444a-b498-78d78042039b service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:19.602397 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce3857-09bc-444a-b498-78d78042039b service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:19.602670 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce3857-09bc-444a-b498-78d78042039b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:19.611677 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce3857-09bc-444a-b498-78d78042039b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:19.613370 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce3857-09bc-444a-b498-78d78042039b service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:19.622973 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1410/2798] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:22:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:22:19.851899 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed856c-5b80-46ba-b49f-dbb62179a1ea None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:19.925185 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed856c-5b80-46ba-b49f-dbb62179a1ea None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:19.930770 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9d4f0-b945-4fe2-83ab-51f419a70552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:19.931469 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9d4f0-b945-4fe2-83ab-51f419a70552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:19.931751 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9d4f0-b945-4fe2-83ab-51f419a70552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:19.932023 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9d4f0-b945-4fe2-83ab-51f419a70552 admin admin] PATH_INFO: `/v3/users/5d17c654bade41c58d2c34372bee5adf` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:19.933174 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa9d4f0-b945-4fe2-83ab-51f419a70552 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d17c654bade41c58d2c34372bee5adf)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:19.935272 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa9d4f0-b945-4fe2-83ab-51f419a70552 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:20.140858 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0aa9d4f0-b945-4fe2-83ab-51f419a70552 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d17c654bade41c58d2c34372bee5adf'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:20.212058 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0aa9d4f0-b945-4fe2-83ab-51f419a70552 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d17c654bade41c58d2c34372bee5adf'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:20.212690 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0aa9d4f0-b945-4fe2-83ab-51f419a70552 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d17c654bade41c58d2c34372bee5adf'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:20.213305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0aa9d4f0-b945-4fe2-83ab-51f419a70552 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d17c654bade41c58d2c34372bee5adf'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:20.221452 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0aa9d4f0-b945-4fe2-83ab-51f419a70552 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d17c654bade41c58d2c34372bee5adf'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:20.239794 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1389/2799] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:22:19 2022] DELETE /identity/v3/users/5d17c654bade41c58d2c34372bee5adf => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:20.244298 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fca441-faf6-438f-a157-a17c3f98096c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:20.245316 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fca441-faf6-438f-a157-a17c3f98096c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:20.245316 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fca441-faf6-438f-a157-a17c3f98096c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:20.264270 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-25fca441-faf6-438f-a157-a17c3f98096c None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:22:20.266049 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-25fca441-faf6-438f-a157-a17c3f98096c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:20.278533 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:22:20.325501 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1411/2800] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:22:20.335766 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9d4f0-b945-4fe2-83ab-51f419a70552 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:20.337205 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0aa9d4f0-b945-4fe2-83ab-51f419a70552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:20.352377 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:22:20.394736 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9d4f0-b945-4fe2-83ab-51f419a70552 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:20.398288 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5abf78-5e60-4e3e-b273-301ccddb2c3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:20.398889 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5abf78-5e60-4e3e-b273-301ccddb2c3b service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:20.399166 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5abf78-5e60-4e3e-b273-301ccddb2c3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:20.399466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5abf78-5e60-4e3e-b273-301ccddb2c3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:20.409157 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5abf78-5e60-4e3e-b273-301ccddb2c3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:20.410573 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5abf78-5e60-4e3e-b273-301ccddb2c3b service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:20.419756 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1390/2801] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:22:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 13:22:20.779010 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fca441-faf6-438f-a157-a17c3f98096c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:20.781115 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-25fca441-faf6-438f-a157-a17c3f98096c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:20.793108 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fca441-faf6-438f-a157-a17c3f98096c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:20.798740 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69d97e-f395-4388-ae1a-c382ab6b592b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:20.799599 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69d97e-f395-4388-ae1a-c382ab6b592b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:20.799881 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69d97e-f395-4388-ae1a-c382ab6b592b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:20.800152 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69d97e-f395-4388-ae1a-c382ab6b592b admin admin] PATH_INFO: `/v3/projects/97f38fc27a764f499abdd2133d0c4a4b` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:20.802383 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f69d97e-f395-4388-ae1a-c382ab6b592b admin admin] RBAC: Authorizing `identity:delete_project(project_id=97f38fc27a764f499abdd2133d0c4a4b)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:20.804446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f69d97e-f395-4388-ae1a-c382ab6b592b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:20.894206 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3f69d97e-f395-4388-ae1a-c382ab6b592b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97f38fc27a764f499abdd2133d0c4a4b', 'request_id': 'req-3f69d97e-f395-4388-ae1a-c382ab6b592b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:20.899799 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3f69d97e-f395-4388-ae1a-c382ab6b592b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97f38fc27a764f499abdd2133d0c4a4b', 'request_id': 'req-3f69d97e-f395-4388-ae1a-c382ab6b592b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:20.965667 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3f69d97e-f395-4388-ae1a-c382ab6b592b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97f38fc27a764f499abdd2133d0c4a4b', 'request_id': 'req-3f69d97e-f395-4388-ae1a-c382ab6b592b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:20.978371 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3f69d97e-f395-4388-ae1a-c382ab6b592b admin admin] The token cache is being invalidate because project 97f38fc27a764f499abdd2133d0c4a4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:22:20.978659 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3f69d97e-f395-4388-ae1a-c382ab6b592b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:20.979312 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3f69d97e-f395-4388-ae1a-c382ab6b592b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:20.980866 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1412/2802] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:22:20 2022] DELETE /identity/v3/projects/97f38fc27a764f499abdd2133d0c4a4b => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:20.985997 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9fb99-7fd8-479c-84ab-00463a2e4f8b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:20.986462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9fb99-7fd8-479c-84ab-00463a2e4f8b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:20.988840 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9fb99-7fd8-479c-84ab-00463a2e4f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:21.005774 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-84c9fb99-7fd8-479c-84ab-00463a2e4f8b None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:22:21.006980 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-84c9fb99-7fd8-479c-84ab-00463a2e4f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:21.065846 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1391/2803] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 13:22:21.073576 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0edab9a-436f-49b8-8822-497f761457dd None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:21.073865 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0edab9a-436f-49b8-8822-497f761457dd None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:21.074137 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0edab9a-436f-49b8-8822-497f761457dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:21.096376 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b0edab9a-436f-49b8-8822-497f761457dd None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:22:21.098123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b0edab9a-436f-49b8-8822-497f761457dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:21.105896 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1413/2804] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:22:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:22:21.110320 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9fb99-7fd8-479c-84ab-00463a2e4f8b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:21.119742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9fb99-7fd8-479c-84ab-00463a2e4f8b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:21.124228 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59805b-3cfd-4f88-9714-6da67afcbfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:21.125172 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59805b-3cfd-4f88-9714-6da67afcbfb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:21.125457 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59805b-3cfd-4f88-9714-6da67afcbfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:21.125731 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59805b-3cfd-4f88-9714-6da67afcbfb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:21.126372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b59805b-3cfd-4f88-9714-6da67afcbfb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:22:21.126675 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b59805b-3cfd-4f88-9714-6da67afcbfb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:21.128365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b59805b-3cfd-4f88-9714-6da67afcbfb3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:21.139585 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1392/2805] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 13:22:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:22:21.144328 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0edab9a-436f-49b8-8822-497f761457dd None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:21.153483 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0edab9a-436f-49b8-8822-497f761457dd None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:21.158702 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366d712-d05e-436e-b652-d436b3740293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:21.159321 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366d712-d05e-436e-b652-d436b3740293 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:21.159604 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366d712-d05e-436e-b652-d436b3740293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:21.159879 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366d712-d05e-436e-b652-d436b3740293 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:21.160502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b366d712-d05e-436e-b652-d436b3740293 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:21.162371 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b366d712-d05e-436e-b652-d436b3740293 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:21.240813 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1414/2806] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:22:21 2022] POST /identity/v3/projects => generated 365 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:22:21.244870 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59805b-3cfd-4f88-9714-6da67afcbfb3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:21.256008 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59805b-3cfd-4f88-9714-6da67afcbfb3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:21.313153 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e1e5c-6fb5-43a4-a4d3-9086a0aaed90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:21.313768 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e1e5c-6fb5-43a4-a4d3-9086a0aaed90 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:21.314053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e1e5c-6fb5-43a4-a4d3-9086a0aaed90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:21.314326 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e1e5c-6fb5-43a4-a4d3-9086a0aaed90 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:21.314990 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e1e5c-6fb5-43a4-a4d3-9086a0aaed90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:21.316972 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e1e5c-6fb5-43a4-a4d3-9086a0aaed90 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:21.388852 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1393/2807] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:22:21 2022] POST /identity/v3/users => generated 283 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:22:21.393762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366d712-d05e-436e-b652-d436b3740293 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:21.404510 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366d712-d05e-436e-b652-d436b3740293 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:21.409447 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eeda17-4fbe-4ded-b846-3740febd5d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:21.410035 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eeda17-4fbe-4ded-b846-3740febd5d24 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:21.410314 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eeda17-4fbe-4ded-b846-3740febd5d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:21.410588 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eeda17-4fbe-4ded-b846-3740febd5d24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:21.411276 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eeda17-4fbe-4ded-b846-3740febd5d24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:21.412853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eeda17-4fbe-4ded-b846-3740febd5d24 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:21.423352 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1415/2808] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:22:21.427353 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e1e5c-6fb5-43a4-a4d3-9086a0aaed90 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:21.439219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e1e5c-6fb5-43a4-a4d3-9086a0aaed90 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:21.443819 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73c15d-a253-499d-a016-103cdcb74f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:21.444405 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73c15d-a253-499d-a016-103cdcb74f31 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:21.444685 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73c15d-a253-499d-a016-103cdcb74f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:21.444957 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73c15d-a253-499d-a016-103cdcb74f31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:21.445588 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73c15d-a253-499d-a016-103cdcb74f31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:21.447156 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73c15d-a253-499d-a016-103cdcb74f31 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:21.458429 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1394/2809] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:22:21.462442 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eeda17-4fbe-4ded-b846-3740febd5d24 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:21.472613 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eeda17-4fbe-4ded-b846-3740febd5d24 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:21.477588 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984793d8-bed9-40a2-8c15-e0c827cec27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:21.478222 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984793d8-bed9-40a2-8c15-e0c827cec27e admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:21.478500 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984793d8-bed9-40a2-8c15-e0c827cec27e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:21.478774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984793d8-bed9-40a2-8c15-e0c827cec27e admin admin] PATH_INFO: `/v3/projects/ba9c34960b544dce8f687fdd57453b3c/users/2cea21281e9d42e6abc34b5f8fd1c46d/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:21.493095 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984793d8-bed9-40a2-8c15-e0c827cec27e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba9c34960b544dce8f687fdd57453b3c, user_id=2cea21281e9d42e6abc34b5f8fd1c46d, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:21.495709 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984793d8-bed9-40a2-8c15-e0c827cec27e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:21.571524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1416/2810] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:22:21 2022] PUT /identity/v3/projects/ba9c34960b544dce8f687fdd57453b3c/users/2cea21281e9d42e6abc34b5f8fd1c46d/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:21.583543 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf362c-ba46-4f8f-8e58-88da89cc45fe None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:21.584564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf362c-ba46-4f8f-8e58-88da89cc45fe None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:21.584564 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf362c-ba46-4f8f-8e58-88da89cc45fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:21.612910 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-25cf362c-ba46-4f8f-8e58-88da89cc45fe None None] MFA Rules not processed for user `2cea21281e9d42e6abc34b5f8fd1c46d`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:22:21.614130 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-25cf362c-ba46-4f8f-8e58-88da89cc45fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:21.623434 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:22:21.680852 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1395/2811] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:22:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:22:21.685316 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984793d8-bed9-40a2-8c15-e0c827cec27e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:21.727920 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:22:21.747874 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984793d8-bed9-40a2-8c15-e0c827cec27e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:21.753020 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06974349-1a72-4f78-af3e-7a1c3b8b07fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:21.753652 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06974349-1a72-4f78-af3e-7a1c3b8b07fc admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:21.754202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06974349-1a72-4f78-af3e-7a1c3b8b07fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:21.754550 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06974349-1a72-4f78-af3e-7a1c3b8b07fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:21.755212 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06974349-1a72-4f78-af3e-7a1c3b8b07fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:21.757529 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06974349-1a72-4f78-af3e-7a1c3b8b07fc admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:21.836695 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1417/2812] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:22:21 2022] POST /identity/v3/projects => generated 365 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:22:21.840800 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf362c-ba46-4f8f-8e58-88da89cc45fe None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:21.852219 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf362c-ba46-4f8f-8e58-88da89cc45fe None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:21.907590 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435ba33-a744-429d-b859-3d69cfe330ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:21.908189 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435ba33-a744-429d-b859-3d69cfe330ff admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:21.908469 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435ba33-a744-429d-b859-3d69cfe330ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:21.908742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435ba33-a744-429d-b859-3d69cfe330ff admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:21.909365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4435ba33-a744-429d-b859-3d69cfe330ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:21.911295 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4435ba33-a744-429d-b859-3d69cfe330ff admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:21.982551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1396/2813] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:22:21 2022] POST /identity/v3/users => generated 289 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:22:21.987044 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06974349-1a72-4f78-af3e-7a1c3b8b07fc admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:21.997277 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06974349-1a72-4f78-af3e-7a1c3b8b07fc admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:22.002581 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ffdc6-9eec-4210-8ce1-b703665f8c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:22.003116 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ffdc6-9eec-4210-8ce1-b703665f8c46 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:22.003431 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ffdc6-9eec-4210-8ce1-b703665f8c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:22.003706 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ffdc6-9eec-4210-8ce1-b703665f8c46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:22.004337 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ffdc6-9eec-4210-8ce1-b703665f8c46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:22.005927 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ffdc6-9eec-4210-8ce1-b703665f8c46 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:22.016381 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1418/2814] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:22:22.020343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435ba33-a744-429d-b859-3d69cfe330ff admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:22.030392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435ba33-a744-429d-b859-3d69cfe330ff admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:22.035554 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d3949-c6c3-47ca-9bb5-6f8a314aa8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:22.036195 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d3949-c6c3-47ca-9bb5-6f8a314aa8c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:22.036471 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d3949-c6c3-47ca-9bb5-6f8a314aa8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:22.036742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d3949-c6c3-47ca-9bb5-6f8a314aa8c8 admin admin] PATH_INFO: `/v3/projects/063ba89cd63d4f7db330b60027d898cf/users/9b13c7472a004879a8e0f24be28971e2/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:22.052014 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0d3949-c6c3-47ca-9bb5-6f8a314aa8c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=063ba89cd63d4f7db330b60027d898cf, user_id=9b13c7472a004879a8e0f24be28971e2, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:22.054620 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0d3949-c6c3-47ca-9bb5-6f8a314aa8c8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:22.137664 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1397/2815] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:22:22 2022] PUT /identity/v3/projects/063ba89cd63d4f7db330b60027d898cf/users/9b13c7472a004879a8e0f24be28971e2/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:22.142717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ffdc6-9eec-4210-8ce1-b703665f8c46 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:22.153247 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ffdc6-9eec-4210-8ce1-b703665f8c46 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:22.209073 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb53902-f024-4191-bb8f-d289736539da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:22.209676 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb53902-f024-4191-bb8f-d289736539da admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:22.209955 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb53902-f024-4191-bb8f-d289736539da admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:22.210226 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb53902-f024-4191-bb8f-d289736539da admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:22.210881 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb53902-f024-4191-bb8f-d289736539da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:22.212498 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb53902-f024-4191-bb8f-d289736539da admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:22.222022 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1419/2816] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:22:22.226485 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d3949-c6c3-47ca-9bb5-6f8a314aa8c8 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:22.236740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d3949-c6c3-47ca-9bb5-6f8a314aa8c8 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:22.241375 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bae34-2f14-448e-868e-3244d285ea91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:22.242019 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bae34-2f14-448e-868e-3244d285ea91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:22.242305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bae34-2f14-448e-868e-3244d285ea91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:22.242583 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bae34-2f14-448e-868e-3244d285ea91 admin admin] PATH_INFO: `/v3/projects/063ba89cd63d4f7db330b60027d898cf/users/9b13c7472a004879a8e0f24be28971e2/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:22.244532 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570bae34-2f14-448e-868e-3244d285ea91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=063ba89cd63d4f7db330b60027d898cf, user_id=9b13c7472a004879a8e0f24be28971e2, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:22.247299 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570bae34-2f14-448e-868e-3244d285ea91 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:22.258157 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1398/2817] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:22:22 2022] PUT /identity/v3/projects/063ba89cd63d4f7db330b60027d898cf/users/9b13c7472a004879a8e0f24be28971e2/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:22.268658 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05159033-540d-4ba7-8f38-cd2560b66884 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:22.268952 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05159033-540d-4ba7-8f38-cd2560b66884 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:22.269235 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05159033-540d-4ba7-8f38-cd2560b66884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:22.296544 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-05159033-540d-4ba7-8f38-cd2560b66884 None None] MFA Rules not processed for user `9b13c7472a004879a8e0f24be28971e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:22:22.297769 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-05159033-540d-4ba7-8f38-cd2560b66884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:22.365302 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1420/2818] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:22:22 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:22:22.377551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bae34-2f14-448e-868e-3244d285ea91 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:22.379287 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-570bae34-2f14-448e-868e-3244d285ea91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:22.436625 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bae34-2f14-448e-868e-3244d285ea91 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:22.441342 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8648196-df8f-4f5a-aa51-2243c624d4ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:22.441935 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8648196-df8f-4f5a-aa51-2243c624d4ce service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:22.442212 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8648196-df8f-4f5a-aa51-2243c624d4ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:22.442477 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8648196-df8f-4f5a-aa51-2243c624d4ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:22.451326 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8648196-df8f-4f5a-aa51-2243c624d4ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:22.453077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8648196-df8f-4f5a-aa51-2243c624d4ce service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:22.462590 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1399/2819] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:22:22 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:22:22.682329 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05159033-540d-4ba7-8f38-cd2560b66884 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:22.692696 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05159033-540d-4ba7-8f38-cd2560b66884 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:22.696189 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc19478-109d-4f9c-97c1-c2f891bb9ebc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:22.696778 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc19478-109d-4f9c-97c1-c2f891bb9ebc service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:22.697053 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc19478-109d-4f9c-97c1-c2f891bb9ebc service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:22.697323 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc19478-109d-4f9c-97c1-c2f891bb9ebc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:22.729231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:22:22.753421 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc19478-109d-4f9c-97c1-c2f891bb9ebc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:22.754839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc19478-109d-4f9c-97c1-c2f891bb9ebc service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:22.763485 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1421/2820] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:22:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:22:26.146666 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8648196-df8f-4f5a-aa51-2243c624d4ce service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:26.155400 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:22:26.209007 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8648196-df8f-4f5a-aa51-2243c624d4ce service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:26.213513 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6318972e-f93e-43d5-8524-52018b96d024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:26.214136 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318972e-f93e-43d5-8524-52018b96d024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:26.214425 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318972e-f93e-43d5-8524-52018b96d024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:26.214699 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318972e-f93e-43d5-8524-52018b96d024 admin admin] PATH_INFO: `/v3/users/2cea21281e9d42e6abc34b5f8fd1c46d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:26.215846 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6318972e-f93e-43d5-8524-52018b96d024 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cea21281e9d42e6abc34b5f8fd1c46d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:26.218549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6318972e-f93e-43d5-8524-52018b96d024 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:26.389932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6318972e-f93e-43d5-8524-52018b96d024 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cea21281e9d42e6abc34b5f8fd1c46d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:26.462604 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6318972e-f93e-43d5-8524-52018b96d024 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cea21281e9d42e6abc34b5f8fd1c46d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:26.463329 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6318972e-f93e-43d5-8524-52018b96d024 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cea21281e9d42e6abc34b5f8fd1c46d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:26.464025 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6318972e-f93e-43d5-8524-52018b96d024 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cea21281e9d42e6abc34b5f8fd1c46d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:26.471588 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6318972e-f93e-43d5-8524-52018b96d024 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cea21281e9d42e6abc34b5f8fd1c46d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:26.489191 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1400/2821] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:22:26 2022] DELETE /identity/v3/users/2cea21281e9d42e6abc34b5f8fd1c46d => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:26.493727 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1221927-7258-4f27-9750-047d8797051e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:26.494193 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1221927-7258-4f27-9750-047d8797051e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:26.494308 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1221927-7258-4f27-9750-047d8797051e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:26.513341 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e1221927-7258-4f27-9750-047d8797051e None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:22:26.514544 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e1221927-7258-4f27-9750-047d8797051e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:26.572424 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1422/2822] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:22:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:22:26.582670 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6318972e-f93e-43d5-8524-52018b96d024 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:26.584252 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6318972e-f93e-43d5-8524-52018b96d024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:26.641038 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6318972e-f93e-43d5-8524-52018b96d024 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:26.644730 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9751489-9d33-42c8-a074-14616432ca6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:26.645328 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9751489-9d33-42c8-a074-14616432ca6f service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:26.645610 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9751489-9d33-42c8-a074-14616432ca6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:26.645882 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9751489-9d33-42c8-a074-14616432ca6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:26.654727 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9751489-9d33-42c8-a074-14616432ca6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:26.656179 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9751489-9d33-42c8-a074-14616432ca6f service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:26.666172 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1401/2823] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:22:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:22:27.047269 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1221927-7258-4f27-9750-047d8797051e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:27.048863 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e1221927-7258-4f27-9750-047d8797051e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:27.063563 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1221927-7258-4f27-9750-047d8797051e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:27.069952 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4455a-3a60-4476-ad0c-2424182ed8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:27.070672 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4455a-3a60-4476-ad0c-2424182ed8c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:27.070884 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4455a-3a60-4476-ad0c-2424182ed8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:27.071154 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4455a-3a60-4476-ad0c-2424182ed8c9 admin admin] PATH_INFO: `/v3/projects/ba9c34960b544dce8f687fdd57453b3c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:27.072304 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d4455a-3a60-4476-ad0c-2424182ed8c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba9c34960b544dce8f687fdd57453b3c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:27.074237 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d4455a-3a60-4476-ad0c-2424182ed8c9 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:27.188707 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a4d4455a-3a60-4476-ad0c-2424182ed8c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba9c34960b544dce8f687fdd57453b3c', 'request_id': 'req-a4d4455a-3a60-4476-ad0c-2424182ed8c9'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:27.195117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a4d4455a-3a60-4476-ad0c-2424182ed8c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba9c34960b544dce8f687fdd57453b3c', 'request_id': 'req-a4d4455a-3a60-4476-ad0c-2424182ed8c9'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:27.257207 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a4d4455a-3a60-4476-ad0c-2424182ed8c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba9c34960b544dce8f687fdd57453b3c', 'request_id': 'req-a4d4455a-3a60-4476-ad0c-2424182ed8c9'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:27.270271 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a4d4455a-3a60-4476-ad0c-2424182ed8c9 admin admin] The token cache is being invalidate because project ba9c34960b544dce8f687fdd57453b3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:22:27.270562 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a4d4455a-3a60-4476-ad0c-2424182ed8c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:27.271422 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a4d4455a-3a60-4476-ad0c-2424182ed8c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:27.272944 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1423/2824] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:22:27 2022] DELETE /identity/v3/projects/ba9c34960b544dce8f687fdd57453b3c => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:27.276986 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9751489-9d33-42c8-a074-14616432ca6f service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:27.286902 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9751489-9d33-42c8-a074-14616432ca6f service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:27.312230 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:22:27.342527 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20821cc1-b74a-47d8-a044-b775e617cf5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:27.343417 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20821cc1-b74a-47d8-a044-b775e617cf5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:27.343846 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20821cc1-b74a-47d8-a044-b775e617cf5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:27.344125 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20821cc1-b74a-47d8-a044-b775e617cf5c admin admin] PATH_INFO: `/v3/users/9b13c7472a004879a8e0f24be28971e2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:27.345273 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20821cc1-b74a-47d8-a044-b775e617cf5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b13c7472a004879a8e0f24be28971e2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:27.347245 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20821cc1-b74a-47d8-a044-b775e617cf5c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:27.499849 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-20821cc1-b74a-47d8-a044-b775e617cf5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b13c7472a004879a8e0f24be28971e2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:27.577246 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-20821cc1-b74a-47d8-a044-b775e617cf5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b13c7472a004879a8e0f24be28971e2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:27.577939 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-20821cc1-b74a-47d8-a044-b775e617cf5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b13c7472a004879a8e0f24be28971e2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:27.578558 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-20821cc1-b74a-47d8-a044-b775e617cf5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b13c7472a004879a8e0f24be28971e2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:27.586742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-20821cc1-b74a-47d8-a044-b775e617cf5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b13c7472a004879a8e0f24be28971e2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:27.605934 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1402/2825] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:22:27 2022] DELETE /identity/v3/users/9b13c7472a004879a8e0f24be28971e2 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:27.995784 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4455a-3a60-4476-ad0c-2424182ed8c9 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:27.997253 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a4d4455a-3a60-4476-ad0c-2424182ed8c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:28.012404 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4455a-3a60-4476-ad0c-2424182ed8c9 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:28.066656 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:22:28.072386 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338bccd-642d-49fd-8228-ec2435a2aa17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:28.073023 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338bccd-642d-49fd-8228-ec2435a2aa17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:28.076399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338bccd-642d-49fd-8228-ec2435a2aa17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:28.076399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338bccd-642d-49fd-8228-ec2435a2aa17 admin admin] PATH_INFO: `/v3/projects/063ba89cd63d4f7db330b60027d898cf` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:28.076399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338bccd-642d-49fd-8228-ec2435a2aa17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=063ba89cd63d4f7db330b60027d898cf)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:28.076680 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338bccd-642d-49fd-8228-ec2435a2aa17 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:28.166026 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5338bccd-642d-49fd-8228-ec2435a2aa17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '063ba89cd63d4f7db330b60027d898cf', 'request_id': 'req-5338bccd-642d-49fd-8228-ec2435a2aa17'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:28.171806 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5338bccd-642d-49fd-8228-ec2435a2aa17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '063ba89cd63d4f7db330b60027d898cf', 'request_id': 'req-5338bccd-642d-49fd-8228-ec2435a2aa17'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:28.234124 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5338bccd-642d-49fd-8228-ec2435a2aa17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '063ba89cd63d4f7db330b60027d898cf', 'request_id': 'req-5338bccd-642d-49fd-8228-ec2435a2aa17'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:28.245980 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5338bccd-642d-49fd-8228-ec2435a2aa17 admin admin] The token cache is being invalidate because project 063ba89cd63d4f7db330b60027d898cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:22:28.246263 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5338bccd-642d-49fd-8228-ec2435a2aa17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:28.246904 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5338bccd-642d-49fd-8228-ec2435a2aa17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:28.248397 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1424/2826] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:22:27 2022] DELETE /identity/v3/projects/063ba89cd63d4f7db330b60027d898cf => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:28.253644 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d1180-d96c-46d2-b033-4abb84e49626 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:28.253967 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d1180-d96c-46d2-b033-4abb84e49626 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:28.254267 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d1180-d96c-46d2-b033-4abb84e49626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:28.273747 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e30d1180-d96c-46d2-b033-4abb84e49626 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:22:28.274973 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e30d1180-d96c-46d2-b033-4abb84e49626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:28.334264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1403/2827] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:22:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:22:28.343131 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3f575-d61f-4556-b967-35e929f6e413 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:28.343557 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3f575-d61f-4556-b967-35e929f6e413 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:28.343832 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3f575-d61f-4556-b967-35e929f6e413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:28.363883 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f7c3f575-d61f-4556-b967-35e929f6e413 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:22:28.365081 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f7c3f575-d61f-4556-b967-35e929f6e413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:28.372928 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1425/2828] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:22:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:22:28.377735 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d1180-d96c-46d2-b033-4abb84e49626 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:28.387332 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d1180-d96c-46d2-b033-4abb84e49626 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:28.392069 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300da5fc-5caa-4036-b7df-0a22ded5dd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:28.392697 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300da5fc-5caa-4036-b7df-0a22ded5dd05 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:28.392974 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300da5fc-5caa-4036-b7df-0a22ded5dd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:28.393251 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300da5fc-5caa-4036-b7df-0a22ded5dd05 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:28.393932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300da5fc-5caa-4036-b7df-0a22ded5dd05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:22:28.394241 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300da5fc-5caa-4036-b7df-0a22ded5dd05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:28.395881 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300da5fc-5caa-4036-b7df-0a22ded5dd05 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:28.407540 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1404/2829] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 13:22:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:22:28.411898 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3f575-d61f-4556-b967-35e929f6e413 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:28.421015 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3f575-d61f-4556-b967-35e929f6e413 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:28.426083 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6f8b0-c91d-470c-811d-e19682bff44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:28.426664 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6f8b0-c91d-470c-811d-e19682bff44b admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:28.426936 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6f8b0-c91d-470c-811d-e19682bff44b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:28.427227 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6f8b0-c91d-470c-811d-e19682bff44b admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:28.427867 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6f8b0-c91d-470c-811d-e19682bff44b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:28.429717 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6f8b0-c91d-470c-811d-e19682bff44b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:28.515349 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1426/2830] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:22:28 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:22:28.519635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300da5fc-5caa-4036-b7df-0a22ded5dd05 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:28.529650 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300da5fc-5caa-4036-b7df-0a22ded5dd05 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:28.562982 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:22:28.588286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0024549-9eff-4c23-8e48-aef24d9909d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:28.588829 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0024549-9eff-4c23-8e48-aef24d9909d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:28.589116 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0024549-9eff-4c23-8e48-aef24d9909d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:28.589399 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0024549-9eff-4c23-8e48-aef24d9909d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:28.590015 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0024549-9eff-4c23-8e48-aef24d9909d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:28.591968 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0024549-9eff-4c23-8e48-aef24d9909d7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:28.674316 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1405/2831] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:22:28 2022] POST /identity/v3/users => generated 294 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:22:28.678524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6f8b0-c91d-470c-811d-e19682bff44b admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:28.688812 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6f8b0-c91d-470c-811d-e19682bff44b admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:28.694069 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a8556-8217-4080-b58a-e7faa180a439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:28.694655 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a8556-8217-4080-b58a-e7faa180a439 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:28.694930 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a8556-8217-4080-b58a-e7faa180a439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:28.695285 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a8556-8217-4080-b58a-e7faa180a439 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:28.695916 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6a8556-8217-4080-b58a-e7faa180a439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:28.697524 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6a8556-8217-4080-b58a-e7faa180a439 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:28.707327 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1427/2832] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:22:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:22:28.711128 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0024549-9eff-4c23-8e48-aef24d9909d7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:28.721486 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0024549-9eff-4c23-8e48-aef24d9909d7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:28.726266 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf802652-e871-4813-b4cf-b94f909febf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:28.726850 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf802652-e871-4813-b4cf-b94f909febf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:28.727125 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf802652-e871-4813-b4cf-b94f909febf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:28.727426 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf802652-e871-4813-b4cf-b94f909febf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:28.728050 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf802652-e871-4813-b4cf-b94f909febf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:28.729618 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf802652-e871-4813-b4cf-b94f909febf0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:28.740133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1406/2833] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:22:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:22:28.744307 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a8556-8217-4080-b58a-e7faa180a439 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:28.754572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a8556-8217-4080-b58a-e7faa180a439 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:28.759315 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6286b86-f67c-469e-8876-75f714143ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:28.759955 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6286b86-f67c-469e-8876-75f714143ec6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:28.760234 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6286b86-f67c-469e-8876-75f714143ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:28.760536 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6286b86-f67c-469e-8876-75f714143ec6 admin admin] PATH_INFO: `/v3/projects/26312974220348a4aee716797a14f113/users/bbb6644ec4cd4f6bb553c9686c3000e8/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:28.774977 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6286b86-f67c-469e-8876-75f714143ec6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26312974220348a4aee716797a14f113, user_id=bbb6644ec4cd4f6bb553c9686c3000e8, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:28.777629 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6286b86-f67c-469e-8876-75f714143ec6 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:28.845022 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1428/2834] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:22:28 2022] PUT /identity/v3/projects/26312974220348a4aee716797a14f113/users/bbb6644ec4cd4f6bb553c9686c3000e8/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:28.856215 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982f23a-91b2-4a8f-9734-a8a3855c18f8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:28.856503 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982f23a-91b2-4a8f-9734-a8a3855c18f8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:28.856781 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982f23a-91b2-4a8f-9734-a8a3855c18f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:28.885031 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9982f23a-91b2-4a8f-9734-a8a3855c18f8 None None] MFA Rules not processed for user `bbb6644ec4cd4f6bb553c9686c3000e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:22:28.886233 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9982f23a-91b2-4a8f-9734-a8a3855c18f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:28.947551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1407/2835] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:22:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:22:28.952250 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6286b86-f67c-469e-8876-75f714143ec6 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:29.013127 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6286b86-f67c-469e-8876-75f714143ec6 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:29.018278 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc22da-c7f2-4541-bef1-80dd7b1a33f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:29.018885 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc22da-c7f2-4541-bef1-80dd7b1a33f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:29.019168 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc22da-c7f2-4541-bef1-80dd7b1a33f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:29.019483 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc22da-c7f2-4541-bef1-80dd7b1a33f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:29.020116 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc22da-c7f2-4541-bef1-80dd7b1a33f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:29.022002 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc22da-c7f2-4541-bef1-80dd7b1a33f6 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:29.104548 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1429/2836] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:22:28 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:22:29.108662 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982f23a-91b2-4a8f-9734-a8a3855c18f8 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:29.119635 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982f23a-91b2-4a8f-9734-a8a3855c18f8 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:29.175802 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc5bca-bb08-485f-8e7c-b34ef69158a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:29.176401 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc5bca-bb08-485f-8e7c-b34ef69158a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:29.176679 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc5bca-bb08-485f-8e7c-b34ef69158a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:29.176949 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc5bca-bb08-485f-8e7c-b34ef69158a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:29.177569 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc5bca-bb08-485f-8e7c-b34ef69158a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:29.179491 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc5bca-bb08-485f-8e7c-b34ef69158a5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:29.245279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1408/2837] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:22:29 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:22:29.249231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc22da-c7f2-4541-bef1-80dd7b1a33f6 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:29.260288 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc22da-c7f2-4541-bef1-80dd7b1a33f6 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:29.265120 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d14dd-3662-4287-8911-efdcb3c3e194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:29.265726 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d14dd-3662-4287-8911-efdcb3c3e194 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:29.266038 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d14dd-3662-4287-8911-efdcb3c3e194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:29.266318 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d14dd-3662-4287-8911-efdcb3c3e194 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:29.267498 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d14dd-3662-4287-8911-efdcb3c3e194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:29.269324 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d14dd-3662-4287-8911-efdcb3c3e194 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:29.271564 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:22:29.280456 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1430/2838] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:22:29.285141 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc5bca-bb08-485f-8e7c-b34ef69158a5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:29.295361 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc5bca-bb08-485f-8e7c-b34ef69158a5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:29.300179 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3444e-1624-4b2a-9846-fcf7ef759d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:29.300807 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3444e-1624-4b2a-9846-fcf7ef759d46 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:29.301089 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3444e-1624-4b2a-9846-fcf7ef759d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:29.301412 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3444e-1624-4b2a-9846-fcf7ef759d46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:29.302050 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3444e-1624-4b2a-9846-fcf7ef759d46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:29.303686 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3444e-1624-4b2a-9846-fcf7ef759d46 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:29.313727 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1409/2839] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:22:29.317782 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d14dd-3662-4287-8911-efdcb3c3e194 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:29.327656 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d14dd-3662-4287-8911-efdcb3c3e194 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:29.332953 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f41394-67ee-438b-baaf-d306ec91c824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:29.333594 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f41394-67ee-438b-baaf-d306ec91c824 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:29.333875 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f41394-67ee-438b-baaf-d306ec91c824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:29.334149 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f41394-67ee-438b-baaf-d306ec91c824 admin admin] PATH_INFO: `/v3/projects/8b861dab6b974a938eab696828e82843/users/006a14ca654a4e68b8832c392296a9a7/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:29.349282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f41394-67ee-438b-baaf-d306ec91c824 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b861dab6b974a938eab696828e82843, user_id=006a14ca654a4e68b8832c392296a9a7, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:29.351935 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f41394-67ee-438b-baaf-d306ec91c824 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:29.462646 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1431/2840] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:22:29 2022] PUT /identity/v3/projects/8b861dab6b974a938eab696828e82843/users/006a14ca654a4e68b8832c392296a9a7/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:29.473559 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f6939-efe4-4dc7-8a6f-627c22ce1ddc None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:29.474072 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f6939-efe4-4dc7-8a6f-627c22ce1ddc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:29.474168 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f6939-efe4-4dc7-8a6f-627c22ce1ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:29.503123 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ec3f6939-efe4-4dc7-8a6f-627c22ce1ddc None None] MFA Rules not processed for user `006a14ca654a4e68b8832c392296a9a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:22:29.504379 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ec3f6939-efe4-4dc7-8a6f-627c22ce1ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:29.513315 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:22:29.568796 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1410/2841] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:22:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:22:29.573826 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f41394-67ee-438b-baaf-d306ec91c824 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:29.637072 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f41394-67ee-438b-baaf-d306ec91c824 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:29.641714 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8dc1c3-533f-4638-bb7d-2a4135f03ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:29.642965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dc1c3-533f-4638-bb7d-2a4135f03ff2 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:29.643291 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dc1c3-533f-4638-bb7d-2a4135f03ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:29.643566 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dc1c3-533f-4638-bb7d-2a4135f03ff2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:29.644351 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8dc1c3-533f-4638-bb7d-2a4135f03ff2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:29.646231 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8dc1c3-533f-4638-bb7d-2a4135f03ff2 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:29.732539 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1432/2842] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:22:29 2022] POST /identity/v3/projects => generated 387 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:22:29.736595 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f6939-efe4-4dc7-8a6f-627c22ce1ddc None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:29.747226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f6939-efe4-4dc7-8a6f-627c22ce1ddc None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:29.803293 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7139879-e334-4bea-b1d5-8e70987511d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:29.803926 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7139879-e334-4bea-b1d5-8e70987511d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:29.804209 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7139879-e334-4bea-b1d5-8e70987511d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:29.804483 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7139879-e334-4bea-b1d5-8e70987511d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:29.805130 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7139879-e334-4bea-b1d5-8e70987511d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:29.807096 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7139879-e334-4bea-b1d5-8e70987511d6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:29.863076 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1411/2843] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:22:29 2022] POST /identity/v3/users => generated 300 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:22:29.867070 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8dc1c3-533f-4638-bb7d-2a4135f03ff2 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:29.878345 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8dc1c3-533f-4638-bb7d-2a4135f03ff2 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:29.882913 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24facd0-d25c-4e66-aa7c-7b230825805f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:29.883539 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24facd0-d25c-4e66-aa7c-7b230825805f admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:29.883817 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24facd0-d25c-4e66-aa7c-7b230825805f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:29.884092 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24facd0-d25c-4e66-aa7c-7b230825805f admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:29.884723 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24facd0-d25c-4e66-aa7c-7b230825805f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:29.886287 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24facd0-d25c-4e66-aa7c-7b230825805f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:29.896868 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1433/2844] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:22:29.900996 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7139879-e334-4bea-b1d5-8e70987511d6 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:29.911327 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7139879-e334-4bea-b1d5-8e70987511d6 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:29.915901 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820b5f2-fbce-484d-a8e9-1951a5d0226a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:29.916544 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820b5f2-fbce-484d-a8e9-1951a5d0226a admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:29.916824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820b5f2-fbce-484d-a8e9-1951a5d0226a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:29.917096 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820b5f2-fbce-484d-a8e9-1951a5d0226a admin admin] PATH_INFO: `/v3/projects/f64eb221ef324f19bf1fe1a1ed6b8e50/users/395462fad1fd4b1dbd82a3d6863d41d3/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:29.931932 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820b5f2-fbce-484d-a8e9-1951a5d0226a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f64eb221ef324f19bf1fe1a1ed6b8e50, user_id=395462fad1fd4b1dbd82a3d6863d41d3, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:29.934556 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820b5f2-fbce-484d-a8e9-1951a5d0226a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:30.010819 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1412/2845] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:22:29 2022] PUT /identity/v3/projects/f64eb221ef324f19bf1fe1a1ed6b8e50/users/395462fad1fd4b1dbd82a3d6863d41d3/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:30.014895 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24facd0-d25c-4e66-aa7c-7b230825805f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:30.025334 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24facd0-d25c-4e66-aa7c-7b230825805f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:30.086370 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0455a7-0bf2-4ce6-9521-5b49ef6932f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:30.086973 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0455a7-0bf2-4ce6-9521-5b49ef6932f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:30.087284 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0455a7-0bf2-4ce6-9521-5b49ef6932f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:30.087560 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0455a7-0bf2-4ce6-9521-5b49ef6932f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:30.088218 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0455a7-0bf2-4ce6-9521-5b49ef6932f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:30.089807 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0455a7-0bf2-4ce6-9521-5b49ef6932f0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:30.099870 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1434/2846] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:22:30.104392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820b5f2-fbce-484d-a8e9-1951a5d0226a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:30.115702 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820b5f2-fbce-484d-a8e9-1951a5d0226a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:30.120376 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a48951-4b8d-46a4-923f-55de4445f222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:30.121013 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a48951-4b8d-46a4-923f-55de4445f222 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:30.121290 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a48951-4b8d-46a4-923f-55de4445f222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:30.121558 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a48951-4b8d-46a4-923f-55de4445f222 admin admin] PATH_INFO: `/v3/projects/f64eb221ef324f19bf1fe1a1ed6b8e50/users/395462fad1fd4b1dbd82a3d6863d41d3/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:30.123488 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a48951-4b8d-46a4-923f-55de4445f222 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f64eb221ef324f19bf1fe1a1ed6b8e50, user_id=395462fad1fd4b1dbd82a3d6863d41d3, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:30.126098 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a48951-4b8d-46a4-923f-55de4445f222 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:30.136772 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1413/2847] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:22:30 2022] PUT /identity/v3/projects/f64eb221ef324f19bf1fe1a1ed6b8e50/users/395462fad1fd4b1dbd82a3d6863d41d3/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:30.148050 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c6b9e-e73c-4049-b12b-98b79cad0636 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:30.148339 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c6b9e-e73c-4049-b12b-98b79cad0636 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:30.148618 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c6b9e-e73c-4049-b12b-98b79cad0636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:30.177631 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-db0c6b9e-e73c-4049-b12b-98b79cad0636 None None] MFA Rules not processed for user `395462fad1fd4b1dbd82a3d6863d41d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:22:30.178879 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-db0c6b9e-e73c-4049-b12b-98b79cad0636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:30.210602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:22:30.247340 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1435/2848] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:22:30 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:22:30.258546 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a48951-4b8d-46a4-923f-55de4445f222 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:30.260368 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e6a48951-4b8d-46a4-923f-55de4445f222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:30.318207 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a48951-4b8d-46a4-923f-55de4445f222 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:30.321821 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3e964-3c77-432b-9e51-dcad75a69c01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:30.322420 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3e964-3c77-432b-9e51-dcad75a69c01 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:30.322699 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3e964-3c77-432b-9e51-dcad75a69c01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:30.322973 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3e964-3c77-432b-9e51-dcad75a69c01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:30.332170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e3e964-3c77-432b-9e51-dcad75a69c01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:30.333588 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e3e964-3c77-432b-9e51-dcad75a69c01 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:30.342882 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1414/2849] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:22:30 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:22:31.346841 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c6b9e-e73c-4049-b12b-98b79cad0636 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:31.348657 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-db0c6b9e-e73c-4049-b12b-98b79cad0636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:31.360467 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c6b9e-e73c-4049-b12b-98b79cad0636 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:31.364668 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c756e5-b004-41fb-a3c0-92ac87107208 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:31.365262 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c756e5-b004-41fb-a3c0-92ac87107208 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:31.365535 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c756e5-b004-41fb-a3c0-92ac87107208 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:31.365802 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c756e5-b004-41fb-a3c0-92ac87107208 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:31.421263 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c756e5-b004-41fb-a3c0-92ac87107208 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:31.422686 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c756e5-b004-41fb-a3c0-92ac87107208 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:31.432167 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1436/2850] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:22:31 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:22:36.220975 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3e964-3c77-432b-9e51-dcad75a69c01 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:36.230078 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3e964-3c77-432b-9e51-dcad75a69c01 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:36.233931 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507b252-225d-45a8-af13-9729141445b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:36.234552 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507b252-225d-45a8-af13-9729141445b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:36.234857 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507b252-225d-45a8-af13-9729141445b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:36.235141 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507b252-225d-45a8-af13-9729141445b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:36.239330 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:22:36.295372 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9507b252-225d-45a8-af13-9729141445b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:36.296805 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9507b252-225d-45a8-af13-9729141445b7 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:36.306055 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1415/2851] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:22:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:22:39.975609 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c756e5-b004-41fb-a3c0-92ac87107208 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:40.037901 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c756e5-b004-41fb-a3c0-92ac87107208 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:40.042376 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe9b2c-b64c-472a-9d4d-45b8f2df678f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:40.042994 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe9b2c-b64c-472a-9d4d-45b8f2df678f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:40.043303 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe9b2c-b64c-472a-9d4d-45b8f2df678f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:40.043577 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe9b2c-b64c-472a-9d4d-45b8f2df678f admin admin] PATH_INFO: `/v3/users/bbb6644ec4cd4f6bb553c9686c3000e8` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:40.044693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfe9b2c-b64c-472a-9d4d-45b8f2df678f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbb6644ec4cd4f6bb553c9686c3000e8)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:40.046594 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfe9b2c-b64c-472a-9d4d-45b8f2df678f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:40.201264 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bbfe9b2c-b64c-472a-9d4d-45b8f2df678f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbb6644ec4cd4f6bb553c9686c3000e8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:40.208583 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bbfe9b2c-b64c-472a-9d4d-45b8f2df678f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbb6644ec4cd4f6bb553c9686c3000e8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:40.267501 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bbfe9b2c-b64c-472a-9d4d-45b8f2df678f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbb6644ec4cd4f6bb553c9686c3000e8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:40.268119 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bbfe9b2c-b64c-472a-9d4d-45b8f2df678f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbb6644ec4cd4f6bb553c9686c3000e8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:40.268733 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bbfe9b2c-b64c-472a-9d4d-45b8f2df678f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbb6644ec4cd4f6bb553c9686c3000e8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:40.274533 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:22:40.287157 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1437/2852] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:22:39 2022] DELETE /identity/v3/users/bbb6644ec4cd4f6bb553c9686c3000e8 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:40.291617 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b78458-dd62-465f-a3e0-66a3e17f5873 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:40.292033 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b78458-dd62-465f-a3e0-66a3e17f5873 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:40.292168 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b78458-dd62-465f-a3e0-66a3e17f5873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:40.311306 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-60b78458-dd62-465f-a3e0-66a3e17f5873 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:22:40.312518 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-60b78458-dd62-465f-a3e0-66a3e17f5873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:40.370714 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1416/2853] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:22:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:22:40.381196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe9b2c-b64c-472a-9d4d-45b8f2df678f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:40.382602 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bbfe9b2c-b64c-472a-9d4d-45b8f2df678f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:40.437592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe9b2c-b64c-472a-9d4d-45b8f2df678f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:40.441181 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a70b0-6f58-4820-b6f6-83ca4a0a5667 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:40.441772 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a70b0-6f58-4820-b6f6-83ca4a0a5667 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:40.442048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a70b0-6f58-4820-b6f6-83ca4a0a5667 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:40.442316 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a70b0-6f58-4820-b6f6-83ca4a0a5667 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:40.451774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138a70b0-6f58-4820-b6f6-83ca4a0a5667 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:40.453119 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138a70b0-6f58-4820-b6f6-83ca4a0a5667 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:40.463072 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1438/2854] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:22:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:22:40.845905 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b78458-dd62-465f-a3e0-66a3e17f5873 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:40.847761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-60b78458-dd62-465f-a3e0-66a3e17f5873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:40.860468 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b78458-dd62-465f-a3e0-66a3e17f5873 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:40.865188 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878e3e1-22d7-40a5-8079-8e6785837ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:40.865824 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878e3e1-22d7-40a5-8079-8e6785837ef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:40.866107 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878e3e1-22d7-40a5-8079-8e6785837ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:40.866382 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878e3e1-22d7-40a5-8079-8e6785837ef4 admin admin] PATH_INFO: `/v3/projects/26312974220348a4aee716797a14f113` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:40.867571 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e878e3e1-22d7-40a5-8079-8e6785837ef4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26312974220348a4aee716797a14f113)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:40.869513 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e878e3e1-22d7-40a5-8079-8e6785837ef4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:40.964927 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e878e3e1-22d7-40a5-8079-8e6785837ef4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26312974220348a4aee716797a14f113', 'request_id': 'req-e878e3e1-22d7-40a5-8079-8e6785837ef4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:40.970305 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e878e3e1-22d7-40a5-8079-8e6785837ef4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26312974220348a4aee716797a14f113', 'request_id': 'req-e878e3e1-22d7-40a5-8079-8e6785837ef4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:41.027571 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e878e3e1-22d7-40a5-8079-8e6785837ef4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26312974220348a4aee716797a14f113', 'request_id': 'req-e878e3e1-22d7-40a5-8079-8e6785837ef4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:41.039229 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e878e3e1-22d7-40a5-8079-8e6785837ef4 admin admin] The token cache is being invalidate because project 26312974220348a4aee716797a14f113 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:22:41.039527 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e878e3e1-22d7-40a5-8079-8e6785837ef4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:41.040157 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e878e3e1-22d7-40a5-8079-8e6785837ef4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:41.041654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1417/2855] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:22:40 2022] DELETE /identity/v3/projects/26312974220348a4aee716797a14f113 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:41.045637 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a70b0-6f58-4820-b6f6-83ca4a0a5667 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:41.055820 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a70b0-6f58-4820-b6f6-83ca4a0a5667 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:41.110718 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322dc7e-1c3d-4f34-b8bd-a258247a18f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:41.111407 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322dc7e-1c3d-4f34-b8bd-a258247a18f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:41.111675 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322dc7e-1c3d-4f34-b8bd-a258247a18f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:41.111947 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322dc7e-1c3d-4f34-b8bd-a258247a18f1 admin admin] PATH_INFO: `/v3/users/006a14ca654a4e68b8832c392296a9a7` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:41.113055 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6322dc7e-1c3d-4f34-b8bd-a258247a18f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=006a14ca654a4e68b8832c392296a9a7)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:41.115568 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6322dc7e-1c3d-4f34-b8bd-a258247a18f1 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:41.268861 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6322dc7e-1c3d-4f34-b8bd-a258247a18f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '006a14ca654a4e68b8832c392296a9a7'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:41.276022 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6322dc7e-1c3d-4f34-b8bd-a258247a18f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '006a14ca654a4e68b8832c392296a9a7'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:41.329442 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6322dc7e-1c3d-4f34-b8bd-a258247a18f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '006a14ca654a4e68b8832c392296a9a7'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:41.330067 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6322dc7e-1c3d-4f34-b8bd-a258247a18f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '006a14ca654a4e68b8832c392296a9a7'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:41.330680 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6322dc7e-1c3d-4f34-b8bd-a258247a18f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '006a14ca654a4e68b8832c392296a9a7'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:41.348141 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1439/2856] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:22:41 2022] DELETE /identity/v3/users/006a14ca654a4e68b8832c392296a9a7 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:41.765833 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878e3e1-22d7-40a5-8079-8e6785837ef4 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:41.767385 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e878e3e1-22d7-40a5-8079-8e6785837ef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:41.783950 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878e3e1-22d7-40a5-8079-8e6785837ef4 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:41.811011 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:22:41.841611 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f09e5-4372-49ef-b37d-e2375472af42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:41.842249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f09e5-4372-49ef-b37d-e2375472af42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:41.842530 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f09e5-4372-49ef-b37d-e2375472af42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:41.842808 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f09e5-4372-49ef-b37d-e2375472af42 admin admin] PATH_INFO: `/v3/projects/8b861dab6b974a938eab696828e82843` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:41.843979 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f09e5-4372-49ef-b37d-e2375472af42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b861dab6b974a938eab696828e82843)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:41.845878 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f09e5-4372-49ef-b37d-e2375472af42 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:41.953565 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-249f09e5-4372-49ef-b37d-e2375472af42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b861dab6b974a938eab696828e82843', 'request_id': 'req-249f09e5-4372-49ef-b37d-e2375472af42'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:41.959560 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-249f09e5-4372-49ef-b37d-e2375472af42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b861dab6b974a938eab696828e82843', 'request_id': 'req-249f09e5-4372-49ef-b37d-e2375472af42'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:42.016027 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-249f09e5-4372-49ef-b37d-e2375472af42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b861dab6b974a938eab696828e82843', 'request_id': 'req-249f09e5-4372-49ef-b37d-e2375472af42'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:42.027567 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-249f09e5-4372-49ef-b37d-e2375472af42 admin admin] The token cache is being invalidate because project 8b861dab6b974a938eab696828e82843 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:22:42.027884 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-249f09e5-4372-49ef-b37d-e2375472af42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:42.028516 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-249f09e5-4372-49ef-b37d-e2375472af42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:42.030106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1418/2857] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:22:41 2022] DELETE /identity/v3/projects/8b861dab6b974a938eab696828e82843 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:42.034146 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322dc7e-1c3d-4f34-b8bd-a258247a18f1 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:42.035611 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6322dc7e-1c3d-4f34-b8bd-a258247a18f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:42.048751 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322dc7e-1c3d-4f34-b8bd-a258247a18f1 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:42.053959 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:22:42.107327 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eee9ff-5a27-42e2-88f9-d1bfd2efd1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:42.107962 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eee9ff-5a27-42e2-88f9-d1bfd2efd1e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:42.108243 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eee9ff-5a27-42e2-88f9-d1bfd2efd1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:42.108519 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eee9ff-5a27-42e2-88f9-d1bfd2efd1e8 admin admin] PATH_INFO: `/v3/users/395462fad1fd4b1dbd82a3d6863d41d3` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:42.109747 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eee9ff-5a27-42e2-88f9-d1bfd2efd1e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=395462fad1fd4b1dbd82a3d6863d41d3)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:42.111686 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eee9ff-5a27-42e2-88f9-d1bfd2efd1e8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:42.235469 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d4eee9ff-5a27-42e2-88f9-d1bfd2efd1e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '395462fad1fd4b1dbd82a3d6863d41d3'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:42.243520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d4eee9ff-5a27-42e2-88f9-d1bfd2efd1e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '395462fad1fd4b1dbd82a3d6863d41d3'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:42.319015 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d4eee9ff-5a27-42e2-88f9-d1bfd2efd1e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '395462fad1fd4b1dbd82a3d6863d41d3'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:42.319660 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d4eee9ff-5a27-42e2-88f9-d1bfd2efd1e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '395462fad1fd4b1dbd82a3d6863d41d3'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:42.320306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d4eee9ff-5a27-42e2-88f9-d1bfd2efd1e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '395462fad1fd4b1dbd82a3d6863d41d3'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:42.338616 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1440/2858] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:22:42 2022] DELETE /identity/v3/users/395462fad1fd4b1dbd82a3d6863d41d3 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:42.697571 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f09e5-4372-49ef-b37d-e2375472af42 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:42.699117 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-249f09e5-4372-49ef-b37d-e2375472af42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:42.714889 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f09e5-4372-49ef-b37d-e2375472af42 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:42.776104 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4d579-18d9-4fdd-be0b-5bf3c200fcb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:42.776759 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4d579-18d9-4fdd-be0b-5bf3c200fcb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:42.776992 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4d579-18d9-4fdd-be0b-5bf3c200fcb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:42.777261 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4d579-18d9-4fdd-be0b-5bf3c200fcb9 admin admin] PATH_INFO: `/v3/projects/f64eb221ef324f19bf1fe1a1ed6b8e50` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:42.778363 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd4d579-18d9-4fdd-be0b-5bf3c200fcb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f64eb221ef324f19bf1fe1a1ed6b8e50)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:42.780291 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd4d579-18d9-4fdd-be0b-5bf3c200fcb9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:42.913370 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-afd4d579-18d9-4fdd-be0b-5bf3c200fcb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f64eb221ef324f19bf1fe1a1ed6b8e50', 'request_id': 'req-afd4d579-18d9-4fdd-be0b-5bf3c200fcb9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:42.918732 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-afd4d579-18d9-4fdd-be0b-5bf3c200fcb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f64eb221ef324f19bf1fe1a1ed6b8e50', 'request_id': 'req-afd4d579-18d9-4fdd-be0b-5bf3c200fcb9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:42.921306 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:22:42.975975 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-afd4d579-18d9-4fdd-be0b-5bf3c200fcb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f64eb221ef324f19bf1fe1a1ed6b8e50', 'request_id': 'req-afd4d579-18d9-4fdd-be0b-5bf3c200fcb9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:42.987707 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-afd4d579-18d9-4fdd-be0b-5bf3c200fcb9 admin admin] The token cache is being invalidate because project f64eb221ef324f19bf1fe1a1ed6b8e50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:22:42.988009 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-afd4d579-18d9-4fdd-be0b-5bf3c200fcb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:42.988651 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-afd4d579-18d9-4fdd-be0b-5bf3c200fcb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:22:42.990768 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1419/2859] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:22:42 2022] DELETE /identity/v3/projects/f64eb221ef324f19bf1fe1a1ed6b8e50 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:42.997124 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7629c7-944c-4683-b49a-e1c300830064 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:42.997434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7629c7-944c-4683-b49a-e1c300830064 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:42.997888 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7629c7-944c-4683-b49a-e1c300830064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:43.018521 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3b7629c7-944c-4683-b49a-e1c300830064 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:22:43.020048 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3b7629c7-944c-4683-b49a-e1c300830064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:43.084041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1441/2860] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:22:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:22:43.092527 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe50d83-37e1-4ea4-9852-6943777bb522 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:43.092816 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe50d83-37e1-4ea4-9852-6943777bb522 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:43.093314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe50d83-37e1-4ea4-9852-6943777bb522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:43.116412 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4fe50d83-37e1-4ea4-9852-6943777bb522 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:22:43.117647 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4fe50d83-37e1-4ea4-9852-6943777bb522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:43.125863 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1420/2861] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:22:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:22:43.130562 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7629c7-944c-4683-b49a-e1c300830064 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:43.140299 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7629c7-944c-4683-b49a-e1c300830064 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:43.144740 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ba699-9da2-4f01-bef8-d2e66de13f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:43.145355 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ba699-9da2-4f01-bef8-d2e66de13f56 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:43.145634 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ba699-9da2-4f01-bef8-d2e66de13f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:43.145949 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ba699-9da2-4f01-bef8-d2e66de13f56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:43.146615 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ba699-9da2-4f01-bef8-d2e66de13f56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:22:43.146919 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ba699-9da2-4f01-bef8-d2e66de13f56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:43.148517 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ba699-9da2-4f01-bef8-d2e66de13f56 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:43.159835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1442/2862] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 13:22:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:22:43.164483 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe50d83-37e1-4ea4-9852-6943777bb522 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:43.173754 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe50d83-37e1-4ea4-9852-6943777bb522 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:43.179153 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b105d-c2ea-4320-9aba-a98cd0229f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:43.179765 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b105d-c2ea-4320-9aba-a98cd0229f7e admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:43.180045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b105d-c2ea-4320-9aba-a98cd0229f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:43.180314 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b105d-c2ea-4320-9aba-a98cd0229f7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:43.180933 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b105d-c2ea-4320-9aba-a98cd0229f7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:43.182798 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b105d-c2ea-4320-9aba-a98cd0229f7e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:43.297471 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1421/2863] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:22:43 2022] POST /identity/v3/projects => generated 373 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:22:43.302162 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ba699-9da2-4f01-bef8-d2e66de13f56 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:43.312353 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ba699-9da2-4f01-bef8-d2e66de13f56 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:43.324265 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:22:43.370817 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a9a64-0241-4b7e-aa21-c74614e97e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:43.371454 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a9a64-0241-4b7e-aa21-c74614e97e70 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:43.371733 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a9a64-0241-4b7e-aa21-c74614e97e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:43.372007 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a9a64-0241-4b7e-aa21-c74614e97e70 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:43.372634 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a9a64-0241-4b7e-aa21-c74614e97e70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:43.374587 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a9a64-0241-4b7e-aa21-c74614e97e70 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:43.449931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1443/2864] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:22:43 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:22:43.453930 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b105d-c2ea-4320-9aba-a98cd0229f7e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:43.464095 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b105d-c2ea-4320-9aba-a98cd0229f7e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:43.468669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01bd88c-11cc-45bc-8882-a8512a81de34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:43.469260 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01bd88c-11cc-45bc-8882-a8512a81de34 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:43.469538 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01bd88c-11cc-45bc-8882-a8512a81de34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:43.469814 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01bd88c-11cc-45bc-8882-a8512a81de34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:43.470445 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01bd88c-11cc-45bc-8882-a8512a81de34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:43.472348 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01bd88c-11cc-45bc-8882-a8512a81de34 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:43.482191 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1422/2865] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:22:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:22:43.486117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a9a64-0241-4b7e-aa21-c74614e97e70 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:43.497133 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a9a64-0241-4b7e-aa21-c74614e97e70 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:43.501739 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab29c11-9914-4802-bced-d0085821f1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:43.502322 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab29c11-9914-4802-bced-d0085821f1cf admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:43.502600 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab29c11-9914-4802-bced-d0085821f1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:43.502869 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab29c11-9914-4802-bced-d0085821f1cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:43.503538 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab29c11-9914-4802-bced-d0085821f1cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:43.506379 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab29c11-9914-4802-bced-d0085821f1cf admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:43.516176 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1444/2866] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:22:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:22:43.520094 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01bd88c-11cc-45bc-8882-a8512a81de34 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:43.531077 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01bd88c-11cc-45bc-8882-a8512a81de34 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:43.535663 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa2c2d-938a-4726-90df-a2e9fe7da541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:43.536419 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa2c2d-938a-4726-90df-a2e9fe7da541 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:43.536701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa2c2d-938a-4726-90df-a2e9fe7da541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:43.536974 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa2c2d-938a-4726-90df-a2e9fe7da541 admin admin] PATH_INFO: `/v3/projects/c045d5171cae4dc4b806785e29584fa1/users/5ff57022cc1d4daea192fafdfab91ec1/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:43.551246 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa2c2d-938a-4726-90df-a2e9fe7da541 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c045d5171cae4dc4b806785e29584fa1, user_id=5ff57022cc1d4daea192fafdfab91ec1, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:43.554365 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa2c2d-938a-4726-90df-a2e9fe7da541 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:43.627221 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1423/2867] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:22:43 2022] PUT /identity/v3/projects/c045d5171cae4dc4b806785e29584fa1/users/5ff57022cc1d4daea192fafdfab91ec1/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:43.632400 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3314622-d558-4f89-bcb5-7b1d9efc39c1 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:43.632692 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3314622-d558-4f89-bcb5-7b1d9efc39c1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:43.632973 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3314622-d558-4f89-bcb5-7b1d9efc39c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:43.652056 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d3314622-d558-4f89-bcb5-7b1d9efc39c1 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:22:43.653269 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d3314622-d558-4f89-bcb5-7b1d9efc39c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:43.710874 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1445/2868] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:22:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:22:43.720656 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa2c2d-938a-4726-90df-a2e9fe7da541 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:43.722586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-95aa2c2d-938a-4726-90df-a2e9fe7da541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:43.780083 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa2c2d-938a-4726-90df-a2e9fe7da541 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:43.783676 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc48d8d-eae4-4db7-9111-204e4d5b905d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:43.784279 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc48d8d-eae4-4db7-9111-204e4d5b905d service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:43.784557 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc48d8d-eae4-4db7-9111-204e4d5b905d service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:43.784829 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc48d8d-eae4-4db7-9111-204e4d5b905d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:43.794746 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc48d8d-eae4-4db7-9111-204e4d5b905d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:43.796193 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc48d8d-eae4-4db7-9111-204e4d5b905d service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:43.806560 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1424/2869] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:22:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:22:44.933239 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66739f0c-c906-4f81-8a1e-1927f98d7315 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:44.933783 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66739f0c-c906-4f81-8a1e-1927f98d7315 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:44.933783 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66739f0c-c906-4f81-8a1e-1927f98d7315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:44.964973 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-66739f0c-c906-4f81-8a1e-1927f98d7315 None None] MFA Rules not processed for user `5ff57022cc1d4daea192fafdfab91ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:22:44.966199 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-66739f0c-c906-4f81-8a1e-1927f98d7315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:45.031804 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1446/2870] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:22:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:22:45.037011 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc48d8d-eae4-4db7-9111-204e4d5b905d service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:45.046884 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc48d8d-eae4-4db7-9111-204e4d5b905d service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:45.052150 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4213da-f12f-435a-9a0d-04d41159dd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:45.052828 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4213da-f12f-435a-9a0d-04d41159dd16 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:45.053124 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4213da-f12f-435a-9a0d-04d41159dd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:45.053395 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4213da-f12f-435a-9a0d-04d41159dd16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:45.054009 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4213da-f12f-435a-9a0d-04d41159dd16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:45.055922 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4213da-f12f-435a-9a0d-04d41159dd16 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:45.113696 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1425/2871] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:22:45 2022] POST /identity/v3/projects => generated 373 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:22:45.118476 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66739f0c-c906-4f81-8a1e-1927f98d7315 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:45.129452 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66739f0c-c906-4f81-8a1e-1927f98d7315 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:45.150164 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:22:45.186400 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da9a9b-6ebf-48b2-825a-c46c6f10d2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:45.187003 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da9a9b-6ebf-48b2-825a-c46c6f10d2c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:45.187318 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da9a9b-6ebf-48b2-825a-c46c6f10d2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:45.187615 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da9a9b-6ebf-48b2-825a-c46c6f10d2c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:45.188272 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da9a9b-6ebf-48b2-825a-c46c6f10d2c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:45.190255 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da9a9b-6ebf-48b2-825a-c46c6f10d2c6 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:45.249525 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1447/2872] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:22:45 2022] POST /identity/v3/users => generated 293 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:22:45.253555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4213da-f12f-435a-9a0d-04d41159dd16 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:45.265402 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4213da-f12f-435a-9a0d-04d41159dd16 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:45.270119 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52852926-c200-4a1a-8b0b-71b18d43587b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:45.271401 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52852926-c200-4a1a-8b0b-71b18d43587b admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:45.271653 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52852926-c200-4a1a-8b0b-71b18d43587b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:45.271924 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52852926-c200-4a1a-8b0b-71b18d43587b admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:45.272555 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52852926-c200-4a1a-8b0b-71b18d43587b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:45.274134 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52852926-c200-4a1a-8b0b-71b18d43587b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:45.286751 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1426/2873] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:22:45.290923 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da9a9b-6ebf-48b2-825a-c46c6f10d2c6 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:45.301043 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da9a9b-6ebf-48b2-825a-c46c6f10d2c6 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:45.306184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad5c7b-dd4c-48bf-8ec0-137c1fc8867a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:45.306832 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad5c7b-dd4c-48bf-8ec0-137c1fc8867a admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:45.307112 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad5c7b-dd4c-48bf-8ec0-137c1fc8867a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:45.307419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad5c7b-dd4c-48bf-8ec0-137c1fc8867a admin admin] PATH_INFO: `/v3/projects/a5b51c9015f24d89aa64a4cd6e8642f0/users/b102af2be6ed4c14879ba259e6af7d4a/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:45.323408 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad5c7b-dd4c-48bf-8ec0-137c1fc8867a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5b51c9015f24d89aa64a4cd6e8642f0, user_id=b102af2be6ed4c14879ba259e6af7d4a, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:45.326055 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad5c7b-dd4c-48bf-8ec0-137c1fc8867a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:45.393355 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1448/2874] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:22:45 2022] PUT /identity/v3/projects/a5b51c9015f24d89aa64a4cd6e8642f0/users/b102af2be6ed4c14879ba259e6af7d4a/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:45.398062 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52852926-c200-4a1a-8b0b-71b18d43587b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:45.408070 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52852926-c200-4a1a-8b0b-71b18d43587b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:45.456496 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:22:45.463419 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ddb1f-54a2-4506-bd00-f26ab08c8989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:45.464019 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ddb1f-54a2-4506-bd00-f26ab08c8989 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:45.464300 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ddb1f-54a2-4506-bd00-f26ab08c8989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:45.464575 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ddb1f-54a2-4506-bd00-f26ab08c8989 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:45.465213 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649ddb1f-54a2-4506-bd00-f26ab08c8989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:45.466833 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649ddb1f-54a2-4506-bd00-f26ab08c8989 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:45.476981 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1427/2875] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:22:45.481200 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad5c7b-dd4c-48bf-8ec0-137c1fc8867a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:45.492267 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad5c7b-dd4c-48bf-8ec0-137c1fc8867a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:45.496926 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b4f25-ff73-4234-92bf-203e6dce3565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:45.497559 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b4f25-ff73-4234-92bf-203e6dce3565 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:45.497843 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b4f25-ff73-4234-92bf-203e6dce3565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:45.498120 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b4f25-ff73-4234-92bf-203e6dce3565 admin admin] PATH_INFO: `/v3/projects/a5b51c9015f24d89aa64a4cd6e8642f0/users/b102af2be6ed4c14879ba259e6af7d4a/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:45.500098 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b4f25-ff73-4234-92bf-203e6dce3565 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5b51c9015f24d89aa64a4cd6e8642f0, user_id=b102af2be6ed4c14879ba259e6af7d4a, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:45.502762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b4f25-ff73-4234-92bf-203e6dce3565 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:45.513572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1449/2876] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:22:45 2022] PUT /identity/v3/projects/a5b51c9015f24d89aa64a4cd6e8642f0/users/b102af2be6ed4c14879ba259e6af7d4a/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:22:46.607890 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946965f2-6474-4de4-be6c-103200e6c1c2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:46.607890 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946965f2-6474-4de4-be6c-103200e6c1c2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:46.607890 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946965f2-6474-4de4-be6c-103200e6c1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:46.635130 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-946965f2-6474-4de4-be6c-103200e6c1c2 None None] MFA Rules not processed for user `b102af2be6ed4c14879ba259e6af7d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:22:46.636406 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-946965f2-6474-4de4-be6c-103200e6c1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:46.707361 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1428/2877] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:22:46 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:22:46.719827 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b4f25-ff73-4234-92bf-203e6dce3565 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:46.721808 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ea1b4f25-ff73-4234-92bf-203e6dce3565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:46.781309 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b4f25-ff73-4234-92bf-203e6dce3565 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:46.785041 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90889a80-2ef5-4747-b746-cb869919326a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:46.785908 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90889a80-2ef5-4747-b746-cb869919326a service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:46.785908 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90889a80-2ef5-4747-b746-cb869919326a service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:46.789058 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90889a80-2ef5-4747-b746-cb869919326a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:46.845076 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90889a80-2ef5-4747-b746-cb869919326a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:46.846852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90889a80-2ef5-4747-b746-cb869919326a service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:46.855817 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1450/2878] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:22:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:22:48.510459 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946965f2-6474-4de4-be6c-103200e6c1c2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:22:48.511885 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-946965f2-6474-4de4-be6c-103200e6c1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:22:48.524389 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946965f2-6474-4de4-be6c-103200e6c1c2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:22:48.527954 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ce77c-dc6a-4fa9-aa9d-eb0334e3fff2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:22:48.528551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ce77c-dc6a-4fa9-aa9d-eb0334e3fff2 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:22:48.528835 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ce77c-dc6a-4fa9-aa9d-eb0334e3fff2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:22:48.529112 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ce77c-dc6a-4fa9-aa9d-eb0334e3fff2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:22:48.538653 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523ce77c-dc6a-4fa9-aa9d-eb0334e3fff2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:22:48.540059 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523ce77c-dc6a-4fa9-aa9d-eb0334e3fff2 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:22:48.549512 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1429/2879] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:22:48 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:22:56.851840 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=81,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:22:58.545275 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=44 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:23:03.877884 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90889a80-2ef5-4747-b746-cb869919326a service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:03.944465 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90889a80-2ef5-4747-b746-cb869919326a service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:03.948996 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac99950-19fd-4a2e-84e9-abc3ebac9931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:03.949621 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac99950-19fd-4a2e-84e9-abc3ebac9931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:03.949905 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac99950-19fd-4a2e-84e9-abc3ebac9931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:03.950184 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac99950-19fd-4a2e-84e9-abc3ebac9931 admin admin] PATH_INFO: `/v3/users/5ff57022cc1d4daea192fafdfab91ec1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:03.951336 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac99950-19fd-4a2e-84e9-abc3ebac9931 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ff57022cc1d4daea192fafdfab91ec1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:03.953258 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac99950-19fd-4a2e-84e9-abc3ebac9931 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:04.136063 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-eac99950-19fd-4a2e-84e9-abc3ebac9931 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ff57022cc1d4daea192fafdfab91ec1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:04.142881 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-eac99950-19fd-4a2e-84e9-abc3ebac9931 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ff57022cc1d4daea192fafdfab91ec1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:04.202908 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-eac99950-19fd-4a2e-84e9-abc3ebac9931 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ff57022cc1d4daea192fafdfab91ec1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:04.203655 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-eac99950-19fd-4a2e-84e9-abc3ebac9931 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ff57022cc1d4daea192fafdfab91ec1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:04.204379 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-eac99950-19fd-4a2e-84e9-abc3ebac9931 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ff57022cc1d4daea192fafdfab91ec1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:04.224259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1451/2880] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:23:03 2022] DELETE /identity/v3/users/5ff57022cc1d4daea192fafdfab91ec1 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:23:04.473283 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ce77c-dc6a-4fa9-aa9d-eb0334e3fff2 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:04.486881 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ce77c-dc6a-4fa9-aa9d-eb0334e3fff2 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:04.557345 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef9efd-a988-4632-94a1-eb878d213411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:04.557971 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef9efd-a988-4632-94a1-eb878d213411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:04.558252 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef9efd-a988-4632-94a1-eb878d213411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:04.558524 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef9efd-a988-4632-94a1-eb878d213411 admin admin] PATH_INFO: `/v3/projects/c045d5171cae4dc4b806785e29584fa1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:04.559761 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbef9efd-a988-4632-94a1-eb878d213411 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c045d5171cae4dc4b806785e29584fa1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:04.561659 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbef9efd-a988-4632-94a1-eb878d213411 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:04.660053 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-dbef9efd-a988-4632-94a1-eb878d213411 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c045d5171cae4dc4b806785e29584fa1', 'request_id': 'req-dbef9efd-a988-4632-94a1-eb878d213411'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:04.665835 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-dbef9efd-a988-4632-94a1-eb878d213411 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c045d5171cae4dc4b806785e29584fa1', 'request_id': 'req-dbef9efd-a988-4632-94a1-eb878d213411'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:04.745529 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-dbef9efd-a988-4632-94a1-eb878d213411 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c045d5171cae4dc4b806785e29584fa1', 'request_id': 'req-dbef9efd-a988-4632-94a1-eb878d213411'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:04.757548 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-dbef9efd-a988-4632-94a1-eb878d213411 admin admin] The token cache is being invalidate because project c045d5171cae4dc4b806785e29584fa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:23:04.758553 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-dbef9efd-a988-4632-94a1-eb878d213411 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:04.758553 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-dbef9efd-a988-4632-94a1-eb878d213411 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:04.760133 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1430/2881] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:23:04 2022] DELETE /identity/v3/projects/c045d5171cae4dc4b806785e29584fa1 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:23:04.764104 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac99950-19fd-4a2e-84e9-abc3ebac9931 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:04.765520 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-eac99950-19fd-4a2e-84e9-abc3ebac9931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:04.779893 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac99950-19fd-4a2e-84e9-abc3ebac9931 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:04.837129 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f46e2-686e-45cf-9335-7a354ad104ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:04.837780 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f46e2-686e-45cf-9335-7a354ad104ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:04.838060 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f46e2-686e-45cf-9335-7a354ad104ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:04.838363 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f46e2-686e-45cf-9335-7a354ad104ab admin admin] PATH_INFO: `/v3/users/b102af2be6ed4c14879ba259e6af7d4a` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:04.839542 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f46e2-686e-45cf-9335-7a354ad104ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=b102af2be6ed4c14879ba259e6af7d4a)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:04.841504 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f46e2-686e-45cf-9335-7a354ad104ab admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:04.863941 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:23:05.051762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b35f46e2-686e-45cf-9335-7a354ad104ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b102af2be6ed4c14879ba259e6af7d4a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:05.059786 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b35f46e2-686e-45cf-9335-7a354ad104ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b102af2be6ed4c14879ba259e6af7d4a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:05.116958 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b35f46e2-686e-45cf-9335-7a354ad104ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b102af2be6ed4c14879ba259e6af7d4a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:05.117581 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b35f46e2-686e-45cf-9335-7a354ad104ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b102af2be6ed4c14879ba259e6af7d4a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:05.118186 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b35f46e2-686e-45cf-9335-7a354ad104ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b102af2be6ed4c14879ba259e6af7d4a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:05.136680 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1452/2882] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:23:04 2022] DELETE /identity/v3/users/b102af2be6ed4c14879ba259e6af7d4a => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:23:05.370908 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef9efd-a988-4632-94a1-eb878d213411 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:05.372366 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-dbef9efd-a988-4632-94a1-eb878d213411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:05.386002 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef9efd-a988-4632-94a1-eb878d213411 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:05.445781 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11baaafb-b1d9-459f-9095-5209b56cc857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:05.446411 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11baaafb-b1d9-459f-9095-5209b56cc857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:05.446688 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11baaafb-b1d9-459f-9095-5209b56cc857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:05.446961 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11baaafb-b1d9-459f-9095-5209b56cc857 admin admin] PATH_INFO: `/v3/projects/a5b51c9015f24d89aa64a4cd6e8642f0` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:05.448142 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11baaafb-b1d9-459f-9095-5209b56cc857 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5b51c9015f24d89aa64a4cd6e8642f0)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:05.450035 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11baaafb-b1d9-459f-9095-5209b56cc857 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:05.528669 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:23:05.540477 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-11baaafb-b1d9-459f-9095-5209b56cc857 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5b51c9015f24d89aa64a4cd6e8642f0', 'request_id': 'req-11baaafb-b1d9-459f-9095-5209b56cc857'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:05.545938 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-11baaafb-b1d9-459f-9095-5209b56cc857 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5b51c9015f24d89aa64a4cd6e8642f0', 'request_id': 'req-11baaafb-b1d9-459f-9095-5209b56cc857'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:05.625709 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-11baaafb-b1d9-459f-9095-5209b56cc857 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5b51c9015f24d89aa64a4cd6e8642f0', 'request_id': 'req-11baaafb-b1d9-459f-9095-5209b56cc857'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:05.637577 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-11baaafb-b1d9-459f-9095-5209b56cc857 admin admin] The token cache is being invalidate because project a5b51c9015f24d89aa64a4cd6e8642f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:23:05.637922 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-11baaafb-b1d9-459f-9095-5209b56cc857 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:05.638580 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-11baaafb-b1d9-459f-9095-5209b56cc857 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:05.640084 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1431/2883] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:23:05 2022] DELETE /identity/v3/projects/a5b51c9015f24d89aa64a4cd6e8642f0 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:23:05.645492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b3ea1-fa9c-4128-8823-800f9929ad42 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:05.645847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b3ea1-fa9c-4128-8823-800f9929ad42 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:05.646203 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b3ea1-fa9c-4128-8823-800f9929ad42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:05.666774 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-906b3ea1-fa9c-4128-8823-800f9929ad42 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:23:05.668025 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-906b3ea1-fa9c-4128-8823-800f9929ad42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:05.727531 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1453/2884] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:23:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:23:05.735609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad8c54-b0cf-4550-b884-c30d9507e95e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:05.735897 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad8c54-b0cf-4550-b884-c30d9507e95e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:05.736170 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad8c54-b0cf-4550-b884-c30d9507e95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:05.757778 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-faad8c54-b0cf-4550-b884-c30d9507e95e None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:23:05.759513 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-faad8c54-b0cf-4550-b884-c30d9507e95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:05.767359 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1432/2885] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:23:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:23:05.771846 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b3ea1-fa9c-4128-8823-800f9929ad42 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:05.781544 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b3ea1-fa9c-4128-8823-800f9929ad42 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:05.786140 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774ac35-7719-4b11-a130-e57b8b482a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:05.786736 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774ac35-7719-4b11-a130-e57b8b482a86 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:05.787132 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774ac35-7719-4b11-a130-e57b8b482a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:05.787444 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774ac35-7719-4b11-a130-e57b8b482a86 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:05.788084 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774ac35-7719-4b11-a130-e57b8b482a86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:23:05.788391 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774ac35-7719-4b11-a130-e57b8b482a86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:05.790024 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774ac35-7719-4b11-a130-e57b8b482a86 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:05.801711 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1454/2886] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 13:23:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 13:23:05.806753 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad8c54-b0cf-4550-b884-c30d9507e95e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:05.816106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad8c54-b0cf-4550-b884-c30d9507e95e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:05.821459 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6c395-b798-45ec-90dd-4e419b0f5c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:05.822079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6c395-b798-45ec-90dd-4e419b0f5c13 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:05.822362 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6c395-b798-45ec-90dd-4e419b0f5c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:05.822638 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6c395-b798-45ec-90dd-4e419b0f5c13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:05.823292 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded6c395-b798-45ec-90dd-4e419b0f5c13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:05.825191 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded6c395-b798-45ec-90dd-4e419b0f5c13 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:05.906952 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1433/2887] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:23:05 2022] POST /identity/v3/projects => generated 369 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 13:23:05.911045 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774ac35-7719-4b11-a130-e57b8b482a86 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:05.921123 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774ac35-7719-4b11-a130-e57b8b482a86 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:05.976644 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353941e-42ef-42b6-82c7-e0c5803d20db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:05.977249 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353941e-42ef-42b6-82c7-e0c5803d20db admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:05.977532 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353941e-42ef-42b6-82c7-e0c5803d20db admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:05.977808 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353941e-42ef-42b6-82c7-e0c5803d20db admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:05.978428 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8353941e-42ef-42b6-82c7-e0c5803d20db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:05.980366 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8353941e-42ef-42b6-82c7-e0c5803d20db admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:06.060135 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1455/2888] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:23:05 2022] POST /identity/v3/users => generated 285 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:23:06.064253 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6c395-b798-45ec-90dd-4e419b0f5c13 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:06.074758 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6c395-b798-45ec-90dd-4e419b0f5c13 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:06.081185 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f9433-5dbf-4207-92eb-8b4a5a49b29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:06.081185 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f9433-5dbf-4207-92eb-8b4a5a49b29f admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:06.081185 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f9433-5dbf-4207-92eb-8b4a5a49b29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:06.081185 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f9433-5dbf-4207-92eb-8b4a5a49b29f admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:06.081590 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f9433-5dbf-4207-92eb-8b4a5a49b29f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:06.082665 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f9433-5dbf-4207-92eb-8b4a5a49b29f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:06.093552 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1434/2889] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:23:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:23:06.097853 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353941e-42ef-42b6-82c7-e0c5803d20db admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:06.108909 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353941e-42ef-42b6-82c7-e0c5803d20db admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:06.113698 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d8055-5e96-46e0-9d7b-7cee47c0b7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:06.114273 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d8055-5e96-46e0-9d7b-7cee47c0b7fc admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:06.114555 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d8055-5e96-46e0-9d7b-7cee47c0b7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:06.114831 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d8055-5e96-46e0-9d7b-7cee47c0b7fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:06.116207 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8d8055-5e96-46e0-9d7b-7cee47c0b7fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:06.117794 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8d8055-5e96-46e0-9d7b-7cee47c0b7fc admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:06.128404 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1456/2890] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:23:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:23:06.132835 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f9433-5dbf-4207-92eb-8b4a5a49b29f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:06.142617 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f9433-5dbf-4207-92eb-8b4a5a49b29f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:06.147489 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d35a0-2187-4f6f-88d6-fad63df4c8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:06.148124 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d35a0-2187-4f6f-88d6-fad63df4c8e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:06.148402 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d35a0-2187-4f6f-88d6-fad63df4c8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:06.148675 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d35a0-2187-4f6f-88d6-fad63df4c8e1 admin admin] PATH_INFO: `/v3/projects/446d9ece2433482db243195083e2245f/users/7dc2386471d34b6284934e91b231f317/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:06.163257 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933d35a0-2187-4f6f-88d6-fad63df4c8e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=446d9ece2433482db243195083e2245f, user_id=7dc2386471d34b6284934e91b231f317, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:06.165828 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933d35a0-2187-4f6f-88d6-fad63df4c8e1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:06.254358 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1435/2891] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:23:06 2022] PUT /identity/v3/projects/446d9ece2433482db243195083e2245f/users/7dc2386471d34b6284934e91b231f317/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:23:06.266347 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541ad4f-4bea-4d9e-bf5e-9476d3617f4f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:06.266630 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541ad4f-4bea-4d9e-bf5e-9476d3617f4f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:06.266902 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541ad4f-4bea-4d9e-bf5e-9476d3617f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:06.276256 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:23:06.295947 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d541ad4f-4bea-4d9e-bf5e-9476d3617f4f None None] MFA Rules not processed for user `7dc2386471d34b6284934e91b231f317`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:23:06.306484 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d541ad4f-4bea-4d9e-bf5e-9476d3617f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:06.361225 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1457/2892] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:23:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:23:06.373546 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d35a0-2187-4f6f-88d6-fad63df4c8e1 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:06.375270 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-933d35a0-2187-4f6f-88d6-fad63df4c8e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:06.432585 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d35a0-2187-4f6f-88d6-fad63df4c8e1 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:06.436168 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f66be-2536-4781-bdf0-6392ccd81cb3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35dd27fd356241b885706375d880222f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:06.436802 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f66be-2536-4781-bdf0-6392ccd81cb3 service glance] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:06.437084 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f66be-2536-4781-bdf0-6392ccd81cb3 service glance] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:06.437362 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f66be-2536-4781-bdf0-6392ccd81cb3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:06.446331 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f66be-2536-4781-bdf0-6392ccd81cb3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:06.447798 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f66be-2536-4781-bdf0-6392ccd81cb3 service glance] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:06.456550 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1436/2893] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:23:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:23:16.385628 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=30 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:23:16.452569 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=65,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:23:42.341911 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541ad4f-4bea-4d9e-bf5e-9476d3617f4f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:42.403472 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541ad4f-4bea-4d9e-bf5e-9476d3617f4f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:42.408013 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2542b-445d-4732-bdd1-adc0c2bc34d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:42.408640 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2542b-445d-4732-bdd1-adc0c2bc34d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:42.408923 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2542b-445d-4732-bdd1-adc0c2bc34d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:42.409202 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2542b-445d-4732-bdd1-adc0c2bc34d5 admin admin] PATH_INFO: `/v3/users/7dc2386471d34b6284934e91b231f317` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:42.410323 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2542b-445d-4732-bdd1-adc0c2bc34d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dc2386471d34b6284934e91b231f317)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:42.412283 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2542b-445d-4732-bdd1-adc0c2bc34d5 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:42.567538 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-44f2542b-445d-4732-bdd1-adc0c2bc34d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dc2386471d34b6284934e91b231f317'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:42.574343 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-44f2542b-445d-4732-bdd1-adc0c2bc34d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dc2386471d34b6284934e91b231f317'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:42.623292 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-44f2542b-445d-4732-bdd1-adc0c2bc34d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dc2386471d34b6284934e91b231f317'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:42.623936 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-44f2542b-445d-4732-bdd1-adc0c2bc34d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dc2386471d34b6284934e91b231f317'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:42.624540 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-44f2542b-445d-4732-bdd1-adc0c2bc34d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dc2386471d34b6284934e91b231f317'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:42.641989 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1458/2894] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:23:42 2022] DELETE /identity/v3/users/7dc2386471d34b6284934e91b231f317 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:23:42.646446 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a48cc-2e79-46db-822a-2481af8d8804 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:42.646928 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a48cc-2e79-46db-822a-2481af8d8804 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:42.647062 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a48cc-2e79-46db-822a-2481af8d8804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:42.670254 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-719a48cc-2e79-46db-822a-2481af8d8804 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:23:42.671535 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-719a48cc-2e79-46db-822a-2481af8d8804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:42.731569 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1437/2895] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:23:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:23:42.741229 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2542b-445d-4732-bdd1-adc0c2bc34d5 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:42.745624 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-44f2542b-445d-4732-bdd1-adc0c2bc34d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:42.809703 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2542b-445d-4732-bdd1-adc0c2bc34d5 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:42.813399 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcacaa87-e768-46ba-a324-d521bb360451 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:42.813998 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcacaa87-e768-46ba-a324-d521bb360451 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:42.814279 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcacaa87-e768-46ba-a324-d521bb360451 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:42.814552 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcacaa87-e768-46ba-a324-d521bb360451 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:42.824282 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcacaa87-e768-46ba-a324-d521bb360451 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:42.825661 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcacaa87-e768-46ba-a324-d521bb360451 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:42.835391 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1459/2896] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:23:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:23:43.241808 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a48cc-2e79-46db-822a-2481af8d8804 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:43.243911 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-719a48cc-2e79-46db-822a-2481af8d8804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:43.257079 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a48cc-2e79-46db-822a-2481af8d8804 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:43.263561 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17cac3c-d3b2-4d9e-8558-a998ce9c48f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:43.264207 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cac3c-d3b2-4d9e-8558-a998ce9c48f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:43.265860 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cac3c-d3b2-4d9e-8558-a998ce9c48f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:43.266151 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cac3c-d3b2-4d9e-8558-a998ce9c48f0 admin admin] PATH_INFO: `/v3/projects/446d9ece2433482db243195083e2245f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:43.267345 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17cac3c-d3b2-4d9e-8558-a998ce9c48f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=446d9ece2433482db243195083e2245f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:43.269258 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17cac3c-d3b2-4d9e-8558-a998ce9c48f0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:43.372363 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e17cac3c-d3b2-4d9e-8558-a998ce9c48f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '446d9ece2433482db243195083e2245f', 'request_id': 'req-e17cac3c-d3b2-4d9e-8558-a998ce9c48f0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:43.377812 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e17cac3c-d3b2-4d9e-8558-a998ce9c48f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '446d9ece2433482db243195083e2245f', 'request_id': 'req-e17cac3c-d3b2-4d9e-8558-a998ce9c48f0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:43.417392 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e17cac3c-d3b2-4d9e-8558-a998ce9c48f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '446d9ece2433482db243195083e2245f', 'request_id': 'req-e17cac3c-d3b2-4d9e-8558-a998ce9c48f0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:43.428335 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e17cac3c-d3b2-4d9e-8558-a998ce9c48f0 admin admin] The token cache is being invalidate because project 446d9ece2433482db243195083e2245f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:23:43.430324 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e17cac3c-d3b2-4d9e-8558-a998ce9c48f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:43.430324 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e17cac3c-d3b2-4d9e-8558-a998ce9c48f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:43.431371 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1438/2897] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:23:43 2022] DELETE /identity/v3/projects/446d9ece2433482db243195083e2245f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 13:23:43.436369 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3014d1-8258-4db2-9c78-f4f9fc163b68 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:43.436852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3014d1-8258-4db2-9c78-f4f9fc163b68 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:43.436947 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3014d1-8258-4db2-9c78-f4f9fc163b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:43.457012 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5f3014d1-8258-4db2-9c78-f4f9fc163b68 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:23:43.458228 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5f3014d1-8258-4db2-9c78-f4f9fc163b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:43.466741 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:23:43.520445 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1460/2898] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:23:43.528563 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce6322-a47a-4209-8a20-9d8a6a43a588 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:43.528856 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce6322-a47a-4209-8a20-9d8a6a43a588 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:43.529137 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce6322-a47a-4209-8a20-9d8a6a43a588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:43.548904 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-84ce6322-a47a-4209-8a20-9d8a6a43a588 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:23:43.550107 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-84ce6322-a47a-4209-8a20-9d8a6a43a588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:43.558654 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1439/2899] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:23:43.562826 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3014d1-8258-4db2-9c78-f4f9fc163b68 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:43.573137 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3014d1-8258-4db2-9c78-f4f9fc163b68 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:43.577830 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49fa72-674e-48d4-a764-c98de3dddbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:43.578418 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49fa72-674e-48d4-a764-c98de3dddbbc admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:43.578693 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49fa72-674e-48d4-a764-c98de3dddbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:43.578961 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49fa72-674e-48d4-a764-c98de3dddbbc admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:43.579632 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49fa72-674e-48d4-a764-c98de3dddbbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:23:43.579931 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49fa72-674e-48d4-a764-c98de3dddbbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:43.581507 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49fa72-674e-48d4-a764-c98de3dddbbc admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:43.593337 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1461/2900] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 13:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:23:43.597465 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce6322-a47a-4209-8a20-9d8a6a43a588 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:43.607190 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce6322-a47a-4209-8a20-9d8a6a43a588 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:43.611823 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2333d44-77a7-4486-8d5a-b971ed63a239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:43.612439 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2333d44-77a7-4486-8d5a-b971ed63a239 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:43.612721 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2333d44-77a7-4486-8d5a-b971ed63a239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:43.612994 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2333d44-77a7-4486-8d5a-b971ed63a239 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:43.613612 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2333d44-77a7-4486-8d5a-b971ed63a239 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:43.615551 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2333d44-77a7-4486-8d5a-b971ed63a239 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:43.688259 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1440/2901] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:23:43 2022] POST /identity/v3/projects => generated 373 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:23:43.692281 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49fa72-674e-48d4-a764-c98de3dddbbc admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:43.702641 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49fa72-674e-48d4-a764-c98de3dddbbc admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:43.759104 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5e4c8-67a3-4c01-9191-cdc29edbfc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:43.759837 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5e4c8-67a3-4c01-9191-cdc29edbfc80 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:43.760118 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5e4c8-67a3-4c01-9191-cdc29edbfc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:43.760391 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5e4c8-67a3-4c01-9191-cdc29edbfc80 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:43.761015 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d5e4c8-67a3-4c01-9191-cdc29edbfc80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:43.762956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d5e4c8-67a3-4c01-9191-cdc29edbfc80 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:43.834575 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1462/2902] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:23:43 2022] POST /identity/v3/users => generated 287 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:23:43.839412 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2333d44-77a7-4486-8d5a-b971ed63a239 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:43.850014 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2333d44-77a7-4486-8d5a-b971ed63a239 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:43.855448 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33859b9b-45e1-4585-a7b8-6e9f0eb780ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:43.856044 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33859b9b-45e1-4585-a7b8-6e9f0eb780ae admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:43.856327 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33859b9b-45e1-4585-a7b8-6e9f0eb780ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:43.856599 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33859b9b-45e1-4585-a7b8-6e9f0eb780ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:43.857236 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33859b9b-45e1-4585-a7b8-6e9f0eb780ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:43.858814 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33859b9b-45e1-4585-a7b8-6e9f0eb780ae admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:43.868816 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1441/2903] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:23:43.872628 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5e4c8-67a3-4c01-9191-cdc29edbfc80 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:43.882446 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5e4c8-67a3-4c01-9191-cdc29edbfc80 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:43.887551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d23146-973b-4bab-b5c0-2758829e5992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:43.888126 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d23146-973b-4bab-b5c0-2758829e5992 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:43.888404 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d23146-973b-4bab-b5c0-2758829e5992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:43.888676 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d23146-973b-4bab-b5c0-2758829e5992 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:43.889309 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d23146-973b-4bab-b5c0-2758829e5992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:43.890875 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d23146-973b-4bab-b5c0-2758829e5992 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:43.902268 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1463/2904] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:23:43.906202 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33859b9b-45e1-4585-a7b8-6e9f0eb780ae admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:43.916566 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33859b9b-45e1-4585-a7b8-6e9f0eb780ae admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:43.921144 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d68191-27dc-43df-ab6a-d9d3b125056f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:43.921774 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d68191-27dc-43df-ab6a-d9d3b125056f admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:43.922052 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d68191-27dc-43df-ab6a-d9d3b125056f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:43.922324 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d68191-27dc-43df-ab6a-d9d3b125056f admin admin] PATH_INFO: `/v3/projects/492cf99a18f14bcd87e0415ba71987dc/users/5b28b4f5c7e14bf4b9d36386ec41e5cc/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:43.936580 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d68191-27dc-43df-ab6a-d9d3b125056f admin admin] RBAC: Authorizing `identity:create_grant(project_id=492cf99a18f14bcd87e0415ba71987dc, user_id=5b28b4f5c7e14bf4b9d36386ec41e5cc, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:43.939286 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d68191-27dc-43df-ab6a-d9d3b125056f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:44.022682 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1442/2905] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:23:43 2022] PUT /identity/v3/projects/492cf99a18f14bcd87e0415ba71987dc/users/5b28b4f5c7e14bf4b9d36386ec41e5cc/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:23:44.035306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6964b-05b1-46ba-8bf9-5ec9094e5137 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:44.035596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6964b-05b1-46ba-8bf9-5ec9094e5137 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:44.035900 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6964b-05b1-46ba-8bf9-5ec9094e5137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:44.064220 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c8d6964b-05b1-46ba-8bf9-5ec9094e5137 None None] MFA Rules not processed for user `5b28b4f5c7e14bf4b9d36386ec41e5cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:23:44.065438 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c8d6964b-05b1-46ba-8bf9-5ec9094e5137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:44.129992 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1464/2906] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:23:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:23:44.141005 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d68191-27dc-43df-ab6a-d9d3b125056f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:44.142685 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c9d68191-27dc-43df-ab6a-d9d3b125056f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:44.165664 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:23:44.200384 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d68191-27dc-43df-ab6a-d9d3b125056f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:44.204447 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ac7a4-aaf8-4a97-bae0-8ed8a08dee75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab5a0d9c39d84cc7a91f5a752dcf2b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:44.205050 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ac7a4-aaf8-4a97-bae0-8ed8a08dee75 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:44.205331 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ac7a4-aaf8-4a97-bae0-8ed8a08dee75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:44.205603 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ac7a4-aaf8-4a97-bae0-8ed8a08dee75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:44.214597 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ac7a4-aaf8-4a97-bae0-8ed8a08dee75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:44.216321 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ac7a4-aaf8-4a97-bae0-8ed8a08dee75 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:44.225425 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1443/2907] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:23:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:23:54.126722 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=93,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:23:54.263143 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=17 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:23:55.363539 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d6964b-05b1-46ba-8bf9-5ec9094e5137 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:55.426277 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d6964b-05b1-46ba-8bf9-5ec9094e5137 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:55.431111 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e35d14-e9f5-4028-b960-52f60926880d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:55.431772 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e35d14-e9f5-4028-b960-52f60926880d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:55.432055 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e35d14-e9f5-4028-b960-52f60926880d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:55.432965 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e35d14-e9f5-4028-b960-52f60926880d admin admin] PATH_INFO: `/v3/users/5b28b4f5c7e14bf4b9d36386ec41e5cc` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:55.434090 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e35d14-e9f5-4028-b960-52f60926880d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b28b4f5c7e14bf4b9d36386ec41e5cc)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:55.436033 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e35d14-e9f5-4028-b960-52f60926880d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:55.591049 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-29e35d14-e9f5-4028-b960-52f60926880d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b28b4f5c7e14bf4b9d36386ec41e5cc'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:55.598101 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-29e35d14-e9f5-4028-b960-52f60926880d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b28b4f5c7e14bf4b9d36386ec41e5cc'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:55.679502 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-29e35d14-e9f5-4028-b960-52f60926880d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b28b4f5c7e14bf4b9d36386ec41e5cc'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:55.680118 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-29e35d14-e9f5-4028-b960-52f60926880d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b28b4f5c7e14bf4b9d36386ec41e5cc'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:55.680754 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-29e35d14-e9f5-4028-b960-52f60926880d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b28b4f5c7e14bf4b9d36386ec41e5cc'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:55.698671 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1465/2908] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:23:55 2022] DELETE /identity/v3/users/5b28b4f5c7e14bf4b9d36386ec41e5cc => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:23:55.703249 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5784bb-6326-4e9a-be55-df1b30e07462 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:55.703742 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5784bb-6326-4e9a-be55-df1b30e07462 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:55.703840 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5784bb-6326-4e9a-be55-df1b30e07462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:55.723676 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8e5784bb-6326-4e9a-be55-df1b30e07462 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:23:55.730214 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8e5784bb-6326-4e9a-be55-df1b30e07462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:55.783987 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1444/2909] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:23:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:23:55.794346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e35d14-e9f5-4028-b960-52f60926880d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:55.795791 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-29e35d14-e9f5-4028-b960-52f60926880d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:55.852374 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e35d14-e9f5-4028-b960-52f60926880d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:55.855996 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7693519-1e99-42fe-91d4-b73f3e667620 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:55.856588 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7693519-1e99-42fe-91d4-b73f3e667620 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:55.856864 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7693519-1e99-42fe-91d4-b73f3e667620 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:55.857136 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7693519-1e99-42fe-91d4-b73f3e667620 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:55.865956 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7693519-1e99-42fe-91d4-b73f3e667620 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:55.867396 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7693519-1e99-42fe-91d4-b73f3e667620 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:55.876761 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1466/2910] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:23:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:23:56.254903 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5784bb-6326-4e9a-be55-df1b30e07462 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:56.254903 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8e5784bb-6326-4e9a-be55-df1b30e07462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:56.268642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5784bb-6326-4e9a-be55-df1b30e07462 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:56.274548 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048a045-4163-4b8f-8955-1c16deca4ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:56.275308 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048a045-4163-4b8f-8955-1c16deca4ca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:56.275752 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048a045-4163-4b8f-8955-1c16deca4ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:56.276099 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048a045-4163-4b8f-8955-1c16deca4ca8 admin admin] PATH_INFO: `/v3/projects/492cf99a18f14bcd87e0415ba71987dc` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:56.279479 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048a045-4163-4b8f-8955-1c16deca4ca8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=492cf99a18f14bcd87e0415ba71987dc)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:56.281462 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048a045-4163-4b8f-8955-1c16deca4ca8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:56.382663 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d048a045-4163-4b8f-8955-1c16deca4ca8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '492cf99a18f14bcd87e0415ba71987dc', 'request_id': 'req-d048a045-4163-4b8f-8955-1c16deca4ca8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:56.388864 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d048a045-4163-4b8f-8955-1c16deca4ca8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '492cf99a18f14bcd87e0415ba71987dc', 'request_id': 'req-d048a045-4163-4b8f-8955-1c16deca4ca8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:56.461910 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d048a045-4163-4b8f-8955-1c16deca4ca8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '492cf99a18f14bcd87e0415ba71987dc', 'request_id': 'req-d048a045-4163-4b8f-8955-1c16deca4ca8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:56.474535 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d048a045-4163-4b8f-8955-1c16deca4ca8 admin admin] The token cache is being invalidate because project 492cf99a18f14bcd87e0415ba71987dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:23:56.474802 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d048a045-4163-4b8f-8955-1c16deca4ca8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:56.476127 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d048a045-4163-4b8f-8955-1c16deca4ca8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:23:56.477073 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1445/2911] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:23:56 2022] DELETE /identity/v3/projects/492cf99a18f14bcd87e0415ba71987dc => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:23:56.482799 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd1c21-bd18-41f1-9a2d-581cd69436eb None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:56.483306 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd1c21-bd18-41f1-9a2d-581cd69436eb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:56.483430 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd1c21-bd18-41f1-9a2d-581cd69436eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:56.503214 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-cebd1c21-bd18-41f1-9a2d-581cd69436eb None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:23:56.504409 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cebd1c21-bd18-41f1-9a2d-581cd69436eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:56.521165 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:23:56.563554 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1467/2912] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:23:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:23:56.571563 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e85957-d7e5-4bdc-83fd-4f5c70d1c850 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:56.571854 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e85957-d7e5-4bdc-83fd-4f5c70d1c850 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:56.572128 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e85957-d7e5-4bdc-83fd-4f5c70d1c850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:56.592991 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-36e85957-d7e5-4bdc-83fd-4f5c70d1c850 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:23:56.594224 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-36e85957-d7e5-4bdc-83fd-4f5c70d1c850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:56.602324 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1446/2913] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:23:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:23:56.606625 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd1c21-bd18-41f1-9a2d-581cd69436eb None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:56.616251 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd1c21-bd18-41f1-9a2d-581cd69436eb None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:56.620917 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae62bde-ed8c-4a5f-b7f5-36bab603b9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:56.621510 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae62bde-ed8c-4a5f-b7f5-36bab603b9bb admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:56.621790 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae62bde-ed8c-4a5f-b7f5-36bab603b9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:56.622062 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae62bde-ed8c-4a5f-b7f5-36bab603b9bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:56.622695 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae62bde-ed8c-4a5f-b7f5-36bab603b9bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 13:23:56.623001 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae62bde-ed8c-4a5f-b7f5-36bab603b9bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:56.624603 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae62bde-ed8c-4a5f-b7f5-36bab603b9bb admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:56.636482 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1468/2914] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 13:23:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 13:23:56.640701 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e85957-d7e5-4bdc-83fd-4f5c70d1c850 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:56.649804 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e85957-d7e5-4bdc-83fd-4f5c70d1c850 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:56.655215 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ad10e-cfdd-4eef-b75f-c57a0d8b1098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:56.655811 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ad10e-cfdd-4eef-b75f-c57a0d8b1098 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:56.656093 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ad10e-cfdd-4eef-b75f-c57a0d8b1098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:56.656377 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ad10e-cfdd-4eef-b75f-c57a0d8b1098 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:56.656989 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946ad10e-cfdd-4eef-b75f-c57a0d8b1098 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:56.658906 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946ad10e-cfdd-4eef-b75f-c57a0d8b1098 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:56.743259 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1447/2915] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:23:56 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:23:56.747406 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae62bde-ed8c-4a5f-b7f5-36bab603b9bb admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:56.757730 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae62bde-ed8c-4a5f-b7f5-36bab603b9bb admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:56.815113 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39500a46-82e7-4190-96ba-2374814ec78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:56.815757 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39500a46-82e7-4190-96ba-2374814ec78e admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:56.816038 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39500a46-82e7-4190-96ba-2374814ec78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:56.816316 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39500a46-82e7-4190-96ba-2374814ec78e admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:56.816948 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39500a46-82e7-4190-96ba-2374814ec78e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:56.818856 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39500a46-82e7-4190-96ba-2374814ec78e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:56.919454 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1469/2916] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:23:56 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:23:56.923506 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ad10e-cfdd-4eef-b75f-c57a0d8b1098 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:56.934568 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ad10e-cfdd-4eef-b75f-c57a0d8b1098 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:56.939216 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a988a3c2-c60c-42a0-92cf-158210eef15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:56.939809 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988a3c2-c60c-42a0-92cf-158210eef15e admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:56.940080 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988a3c2-c60c-42a0-92cf-158210eef15e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:56.940354 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988a3c2-c60c-42a0-92cf-158210eef15e admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:56.941069 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a988a3c2-c60c-42a0-92cf-158210eef15e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:56.942653 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a988a3c2-c60c-42a0-92cf-158210eef15e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:56.953125 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1448/2917] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:23:56.957185 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39500a46-82e7-4190-96ba-2374814ec78e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:56.967545 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39500a46-82e7-4190-96ba-2374814ec78e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:56.972217 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79649352-2806-4f41-8b96-430ac1366c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:56.972804 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79649352-2806-4f41-8b96-430ac1366c14 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:56.973086 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79649352-2806-4f41-8b96-430ac1366c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:56.973359 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79649352-2806-4f41-8b96-430ac1366c14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:56.973990 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79649352-2806-4f41-8b96-430ac1366c14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:56.975619 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79649352-2806-4f41-8b96-430ac1366c14 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:56.986151 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1470/2918] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:23:56.990008 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a988a3c2-c60c-42a0-92cf-158210eef15e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:57.000817 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a988a3c2-c60c-42a0-92cf-158210eef15e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:57.005351 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f00ab-64c6-46d2-ad3e-bdd47f315e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:57.005985 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f00ab-64c6-46d2-ad3e-bdd47f315e3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:57.006264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f00ab-64c6-46d2-ad3e-bdd47f315e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:57.006546 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f00ab-64c6-46d2-ad3e-bdd47f315e3b admin admin] PATH_INFO: `/v3/projects/2a40aefbd17f4dbc911527a917f03976/users/eaa1bdd8c2f34ebc90d7f5b6ec4c2d51/roles/872ba0c4212a4684b539a68fc2c20b25` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:57.022500 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f00ab-64c6-46d2-ad3e-bdd47f315e3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a40aefbd17f4dbc911527a917f03976, user_id=eaa1bdd8c2f34ebc90d7f5b6ec4c2d51, role_id=872ba0c4212a4684b539a68fc2c20b25)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:57.025205 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f00ab-64c6-46d2-ad3e-bdd47f315e3b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:57.091096 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1449/2919] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:23:56 2022] PUT /identity/v3/projects/2a40aefbd17f4dbc911527a917f03976/users/eaa1bdd8c2f34ebc90d7f5b6ec4c2d51/roles/872ba0c4212a4684b539a68fc2c20b25 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:23:57.096424 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51807851-9da2-46bc-ade1-997ffe083243 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:57.101752 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51807851-9da2-46bc-ade1-997ffe083243 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:57.101752 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51807851-9da2-46bc-ade1-997ffe083243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:57.117890 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-51807851-9da2-46bc-ade1-997ffe083243 None None] MFA Rules not processed for user `903d09eed7334b43af767deea1a6bd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:23:57.119094 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-51807851-9da2-46bc-ade1-997ffe083243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:57.177019 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1471/2920] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:23:57.186485 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f00ab-64c6-46d2-ad3e-bdd47f315e3b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:23:57.188211 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-702f00ab-64c6-46d2-ad3e-bdd47f315e3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:23:57.209567 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:23:57.246210 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f00ab-64c6-46d2-ad3e-bdd47f315e3b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:23:57.249912 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1bb522-64c6-476d-bea2-3061de34754f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:23:57.250509 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1bb522-64c6-476d-bea2-3061de34754f service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:23:57.250790 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1bb522-64c6-476d-bea2-3061de34754f service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:23:57.251064 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1bb522-64c6-476d-bea2-3061de34754f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:23:57.261850 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1bb522-64c6-476d-bea2-3061de34754f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:23:57.263344 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1bb522-64c6-476d-bea2-3061de34754f service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:23:57.272830 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1450/2921] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:23:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:24:02.374194 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144c9ba-1d67-488d-94ae-b717c523a361 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:02.375280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144c9ba-1d67-488d-94ae-b717c523a361 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:02.375592 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144c9ba-1d67-488d-94ae-b717c523a361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:02.406648 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b144c9ba-1d67-488d-94ae-b717c523a361 None None] MFA Rules not processed for user `eaa1bdd8c2f34ebc90d7f5b6ec4c2d51`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:02.408185 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b144c9ba-1d67-488d-94ae-b717c523a361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:02.427260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:02.477809 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1472/2922] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:24:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:24:02.483221 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1bb522-64c6-476d-bea2-3061de34754f service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:02.493257 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1bb522-64c6-476d-bea2-3061de34754f service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:02.498149 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19070efe-28fc-44a4-9dc7-50d1a7257940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:02.498762 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19070efe-28fc-44a4-9dc7-50d1a7257940 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:02.499042 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19070efe-28fc-44a4-9dc7-50d1a7257940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:02.499343 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19070efe-28fc-44a4-9dc7-50d1a7257940 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:02.499990 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19070efe-28fc-44a4-9dc7-50d1a7257940 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:02.501903 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19070efe-28fc-44a4-9dc7-50d1a7257940 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:02.584781 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1451/2923] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 13:24:02 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:24:02.589835 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144c9ba-1d67-488d-94ae-b717c523a361 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:02.601343 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144c9ba-1d67-488d-94ae-b717c523a361 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:02.665337 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b45493-8171-416d-b0b8-328fd39d7bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:02.665991 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b45493-8171-416d-b0b8-328fd39d7bf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:02.666270 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b45493-8171-416d-b0b8-328fd39d7bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:02.666545 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b45493-8171-416d-b0b8-328fd39d7bf4 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:02.667201 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b45493-8171-416d-b0b8-328fd39d7bf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:02.669109 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b45493-8171-416d-b0b8-328fd39d7bf4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:02.777941 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1473/2924] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 13:24:02 2022] POST /identity/v3/users => generated 296 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 13:24:02.782164 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19070efe-28fc-44a4-9dc7-50d1a7257940 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:02.792801 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19070efe-28fc-44a4-9dc7-50d1a7257940 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:02.797323 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9760cc7-4c34-408e-861a-4aab1425d236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:02.797914 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9760cc7-4c34-408e-861a-4aab1425d236 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:02.798198 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9760cc7-4c34-408e-861a-4aab1425d236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:02.798470 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9760cc7-4c34-408e-861a-4aab1425d236 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:02.799113 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9760cc7-4c34-408e-861a-4aab1425d236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:02.800730 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9760cc7-4c34-408e-861a-4aab1425d236 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:02.811351 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1452/2925] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:24:02.815458 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b45493-8171-416d-b0b8-328fd39d7bf4 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:02.836770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b45493-8171-416d-b0b8-328fd39d7bf4 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:02.836770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a2fa2-66be-44e3-81ff-a143113caf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:02.836770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a2fa2-66be-44e3-81ff-a143113caf17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:02.836770 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a2fa2-66be-44e3-81ff-a143113caf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:02.837804 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a2fa2-66be-44e3-81ff-a143113caf17 admin admin] PATH_INFO: `/v3/projects/a17be39fe9754ac593beda82dd4a1e54/users/d90e642fcfbc4664988badec25fb8e4a/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:02.849244 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a2fa2-66be-44e3-81ff-a143113caf17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a17be39fe9754ac593beda82dd4a1e54, user_id=d90e642fcfbc4664988badec25fb8e4a, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:02.851923 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a2fa2-66be-44e3-81ff-a143113caf17 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:02.919132 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1474/2926] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:24:02 2022] PUT /identity/v3/projects/a17be39fe9754ac593beda82dd4a1e54/users/d90e642fcfbc4664988badec25fb8e4a/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:24:02.923081 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9760cc7-4c34-408e-861a-4aab1425d236 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:02.933993 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9760cc7-4c34-408e-861a-4aab1425d236 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:02.997604 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e999b4-8968-4f59-bccc-21e60378616a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:02.998222 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e999b4-8968-4f59-bccc-21e60378616a admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:02.999426 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e999b4-8968-4f59-bccc-21e60378616a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:02.999829 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e999b4-8968-4f59-bccc-21e60378616a admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:03.000766 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e999b4-8968-4f59-bccc-21e60378616a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:03.002458 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e999b4-8968-4f59-bccc-21e60378616a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:03.017065 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1453/2927] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 13:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 13:24:03.018181 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a2fa2-66be-44e3-81ff-a143113caf17 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:03.031637 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a2fa2-66be-44e3-81ff-a143113caf17 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:03.037596 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972ac6e-dce1-4b84-899d-96eedf6f962a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:03.037934 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972ac6e-dce1-4b84-899d-96eedf6f962a admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:03.038213 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972ac6e-dce1-4b84-899d-96eedf6f962a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:03.038492 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972ac6e-dce1-4b84-899d-96eedf6f962a admin admin] PATH_INFO: `/v3/projects/a17be39fe9754ac593beda82dd4a1e54/users/d90e642fcfbc4664988badec25fb8e4a/roles/bffbe227cda1418cbb58394ddff6f861` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:03.040684 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f972ac6e-dce1-4b84-899d-96eedf6f962a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a17be39fe9754ac593beda82dd4a1e54, user_id=d90e642fcfbc4664988badec25fb8e4a, role_id=bffbe227cda1418cbb58394ddff6f861)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:03.046317 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f972ac6e-dce1-4b84-899d-96eedf6f962a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:03.060675 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1475/2928] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 13:24:03 2022] PUT /identity/v3/projects/a17be39fe9754ac593beda82dd4a1e54/users/d90e642fcfbc4664988badec25fb8e4a/roles/bffbe227cda1418cbb58394ddff6f861 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:24:08.300370 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e24be2-3d82-4519-a968-d78d66cbb46a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:08.300838 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e24be2-3d82-4519-a968-d78d66cbb46a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:08.300945 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e24be2-3d82-4519-a968-d78d66cbb46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:08.332167 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-47e24be2-3d82-4519-a968-d78d66cbb46a None None] MFA Rules not processed for user `d90e642fcfbc4664988badec25fb8e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:08.333325 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-47e24be2-3d82-4519-a968-d78d66cbb46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:08.409936 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1454/2929] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:24:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:24:08.426978 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972ac6e-dce1-4b84-899d-96eedf6f962a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:08.428847 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f972ac6e-dce1-4b84-899d-96eedf6f962a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:08.511008 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972ac6e-dce1-4b84-899d-96eedf6f962a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:08.517045 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a9ff4-7170-4e19-8c0c-77b51e0d6707 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:08.517614 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a9ff4-7170-4e19-8c0c-77b51e0d6707 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:08.517891 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a9ff4-7170-4e19-8c0c-77b51e0d6707 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:08.518193 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a9ff4-7170-4e19-8c0c-77b51e0d6707 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:08.523863 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:08.577505 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a9ff4-7170-4e19-8c0c-77b51e0d6707 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:08.579102 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a9ff4-7170-4e19-8c0c-77b51e0d6707 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:08.589959 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1476/2930] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:24:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:24:18.406869 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:18.633494 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=24 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:27.522292 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664d00a-b52d-48b5-8cf6-8c1d2945b30a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:27.522798 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664d00a-b52d-48b5-8cf6-8c1d2945b30a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:27.522900 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664d00a-b52d-48b5-8cf6-8c1d2945b30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:27.541229 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6664d00a-b52d-48b5-8cf6-8c1d2945b30a None None] MFA Rules not processed for user `eaa1bdd8c2f34ebc90d7f5b6ec4c2d51`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:24:27.542875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6664d00a-b52d-48b5-8cf6-8c1d2945b30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:27.550642 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1455/2931] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:24:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:24:27.862678 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a9ff4-7170-4e19-8c0c-77b51e0d6707 service nova] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:27.864221 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-040a9ff4-7170-4e19-8c0c-77b51e0d6707 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:27.920952 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a9ff4-7170-4e19-8c0c-77b51e0d6707 service nova] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:27.924547 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5407cef-2370-4661-a37f-0f256e0446ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:27.925152 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5407cef-2370-4661-a37f-0f256e0446ee service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:27.925428 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5407cef-2370-4661-a37f-0f256e0446ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:27.925700 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5407cef-2370-4661-a37f-0f256e0446ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:27.927523 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d5407cef-2370-4661-a37f-0f256e0446ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:27.936901 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5407cef-2370-4661-a37f-0f256e0446ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:27.938309 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5407cef-2370-4661-a37f-0f256e0446ee service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:27.946259 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1477/2932] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:24:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:24:28.621948 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6664d00a-b52d-48b5-8cf6-8c1d2945b30a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:28.623466 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6664d00a-b52d-48b5-8cf6-8c1d2945b30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:28.696258 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6664d00a-b52d-48b5-8cf6-8c1d2945b30a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:28.701269 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c06f4c-daef-426f-ad60-68041a08a852 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff152e22374407eb72caf648a763107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:28.701875 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c06f4c-daef-426f-ad60-68041a08a852 service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:28.702156 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c06f4c-daef-426f-ad60-68041a08a852 service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:28.702427 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c06f4c-daef-426f-ad60-68041a08a852 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:28.704182 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-67c06f4c-daef-426f-ad60-68041a08a852 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:28.716425 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c06f4c-daef-426f-ad60-68041a08a852 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:28.718001 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c06f4c-daef-426f-ad60-68041a08a852 service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:28.727936 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1456/2933] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:24:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:24:31.502543 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5407cef-2370-4661-a37f-0f256e0446ee service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:31.511482 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5407cef-2370-4661-a37f-0f256e0446ee service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:31.515227 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6e94e-d0d0-4719-8db9-aafef6dbcdfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:31.521206 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6e94e-d0d0-4719-8db9-aafef6dbcdfd service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:31.521206 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6e94e-d0d0-4719-8db9-aafef6dbcdfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:31.521206 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6e94e-d0d0-4719-8db9-aafef6dbcdfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:31.521206 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-fed6e94e-d0d0-4719-8db9-aafef6dbcdfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:31.528552 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed6e94e-d0d0-4719-8db9-aafef6dbcdfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:31.529962 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed6e94e-d0d0-4719-8db9-aafef6dbcdfd service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:31.538206 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1478/2934] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:24:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:24:38.723704 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=35 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:41.534419 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=32 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:43.250678 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c06f4c-daef-426f-ad60-68041a08a852 service placement] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:43.264228 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c06f4c-daef-426f-ad60-68041a08a852 service placement] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:43.270156 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138fa8a-e625-4ab7-a4d2-cc360b77b6eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:43.270763 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138fa8a-e625-4ab7-a4d2-cc360b77b6eb service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:43.271045 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138fa8a-e625-4ab7-a4d2-cc360b77b6eb service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:43.271391 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138fa8a-e625-4ab7-a4d2-cc360b77b6eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:43.273141 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5138fa8a-e625-4ab7-a4d2-cc360b77b6eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:43.286878 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5138fa8a-e625-4ab7-a4d2-cc360b77b6eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:43.290068 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5138fa8a-e625-4ab7-a4d2-cc360b77b6eb service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:43.299443 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1457/2935] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:24:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:24:46.290539 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6e94e-d0d0-4719-8db9-aafef6dbcdfd service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:46.299456 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6e94e-d0d0-4719-8db9-aafef6dbcdfd service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:46.303247 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca2e6d-1f94-456a-8680-90fe0a5ba4a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:46.303839 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca2e6d-1f94-456a-8680-90fe0a5ba4a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:46.304113 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca2e6d-1f94-456a-8680-90fe0a5ba4a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:46.304380 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca2e6d-1f94-456a-8680-90fe0a5ba4a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:46.313512 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ca2e6d-1f94-456a-8680-90fe0a5ba4a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:46.315317 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ca2e6d-1f94-456a-8680-90fe0a5ba4a0 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:46.324476 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1479/2936] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:24:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:24:48.514513 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138fa8a-e625-4ab7-a4d2-cc360b77b6eb service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:24:48.525867 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138fa8a-e625-4ab7-a4d2-cc360b77b6eb service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:24:48.532386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f1ebc-c508-444a-903e-8dd88ecd1cc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:24:48.532924 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f1ebc-c508-444a-903e-8dd88ecd1cc0 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:24:48.533201 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f1ebc-c508-444a-903e-8dd88ecd1cc0 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:24:48.533471 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f1ebc-c508-444a-903e-8dd88ecd1cc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:24:48.535247 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e34f1ebc-c508-444a-903e-8dd88ecd1cc0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:24:48.548264 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f1ebc-c508-444a-903e-8dd88ecd1cc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:24:48.549835 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f1ebc-c508-444a-903e-8dd88ecd1cc0 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:24:48.560559 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1458/2937] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:24:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:24:56.320414 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=6 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:24:58.555978 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=12 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:10.680821 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca2e6d-1f94-456a-8680-90fe0a5ba4a0 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:26:10.683071 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a1ca2e6d-1f94-456a-8680-90fe0a5ba4a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:10.710185 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca2e6d-1f94-456a-8680-90fe0a5ba4a0 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:26:10.715471 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0ef12-ba71-4452-8038-957b839b763d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff152e22374407eb72caf648a763107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:26:10.716068 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0ef12-ba71-4452-8038-957b839b763d service placement] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:10.716440 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0ef12-ba71-4452-8038-957b839b763d service placement] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:10.716663 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0ef12-ba71-4452-8038-957b839b763d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:10.718346 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a7b0ef12-ba71-4452-8038-957b839b763d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:10.730509 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b0ef12-ba71-4452-8038-957b839b763d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:26:10.732106 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b0ef12-ba71-4452-8038-957b839b763d service placement] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:26:10.741898 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1480/2938] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:26:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:26:20.737674 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=10 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:26:47.504386 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57712ef-5806-4fd1-9907-d370542c3c3d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:26:47.504861 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57712ef-5806-4fd1-9907-d370542c3c3d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:26:47.504959 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57712ef-5806-4fd1-9907-d370542c3c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:26:47.524352 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f57712ef-5806-4fd1-9907-d370542c3c3d None None] MFA Rules not processed for user `eaa1bdd8c2f34ebc90d7f5b6ec4c2d51`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:26:47.525586 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f57712ef-5806-4fd1-9907-d370542c3c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:26:47.532410 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1459/2939] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:26:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:26:57.519607 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:27:01.303912 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0ef12-ba71-4452-8038-957b839b763d service placement] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:27:01.313505 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0ef12-ba71-4452-8038-957b839b763d service placement] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:27:01.317279 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350f5476-cb8f-4809-b472-8a21eb54c02d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:27:01.321229 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f5476-cb8f-4809-b472-8a21eb54c02d service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:27:01.321229 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f5476-cb8f-4809-b472-8a21eb54c02d service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:27:01.321229 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f5476-cb8f-4809-b472-8a21eb54c02d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:27:01.321229 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-350f5476-cb8f-4809-b472-8a21eb54c02d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:27:01.341842 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350f5476-cb8f-4809-b472-8a21eb54c02d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:27:01.343337 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350f5476-cb8f-4809-b472-8a21eb54c02d service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:27:01.365117 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1481/2940] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:27:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:27:11.362495 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=13 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:42.797523 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acb9b4-01fe-4d12-b701-f53c3d3ecf43 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:42.798149 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acb9b4-01fe-4d12-b701-f53c3d3ecf43 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:42.798149 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acb9b4-01fe-4d12-b701-f53c3d3ecf43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:42.817687 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-03acb9b4-01fe-4d12-b701-f53c3d3ecf43 None None] MFA Rules not processed for user `eaa1bdd8c2f34ebc90d7f5b6ec4c2d51`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 13:28:42.819562 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-03acb9b4-01fe-4d12-b701-f53c3d3ecf43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:42.826373 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1460/2941] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 13:28:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 13:28:47.743762 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350f5476-cb8f-4809-b472-8a21eb54c02d service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:28:47.755572 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350f5476-cb8f-4809-b472-8a21eb54c02d service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:28:47.761989 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e88607-1b31-4624-a2be-8cf07e20050f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:28:47.762675 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e88607-1b31-4624-a2be-8cf07e20050f service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:28:47.763032 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e88607-1b31-4624-a2be-8cf07e20050f service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:28:47.763409 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e88607-1b31-4624-a2be-8cf07e20050f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:28:47.765226 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b5e88607-1b31-4624-a2be-8cf07e20050f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:28:47.778806 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e88607-1b31-4624-a2be-8cf07e20050f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:28:47.780551 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e88607-1b31-4624-a2be-8cf07e20050f service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:28:47.791955 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1482/2942] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:28:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:28:52.812947 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:28:57.786995 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=6 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:20.436970 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03acb9b4-01fe-4d12-b701-f53c3d3ecf43 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:20.447127 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03acb9b4-01fe-4d12-b701-f53c3d3ecf43 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:20.450715 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b54ca-10ef-4731-b427-bf58ca0c0876 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:20.451480 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b54ca-10ef-4731-b427-bf58ca0c0876 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:20.452025 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b54ca-10ef-4731-b427-bf58ca0c0876 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:20.452310 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b54ca-10ef-4731-b427-bf58ca0c0876 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:20.459744 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b54ca-10ef-4731-b427-bf58ca0c0876 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:20.461724 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b54ca-10ef-4731-b427-bf58ca0c0876 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:20.469762 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1461/2943] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:30.466106 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=6 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:32.353663 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e88607-1b31-4624-a2be-8cf07e20050f service nova] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:32.363280 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e88607-1b31-4624-a2be-8cf07e20050f service nova] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:32.366851 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be550e30-9f83-48b0-afe0-2b416bc7993b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:32.368040 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be550e30-9f83-48b0-afe0-2b416bc7993b service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:32.368321 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be550e30-9f83-48b0-afe0-2b416bc7993b service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:32.368655 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be550e30-9f83-48b0-afe0-2b416bc7993b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:32.377091 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be550e30-9f83-48b0-afe0-2b416bc7993b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:32.379852 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be550e30-9f83-48b0-afe0-2b416bc7993b service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:32.388254 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1483/2944] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:33.620410 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b54ca-10ef-4731-b427-bf58ca0c0876 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:33.635434 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b54ca-10ef-4731-b427-bf58ca0c0876 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:33.641328 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a52a7-c20f-4f1b-af51-d331f82d55fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb3f6deffb54467f8b923956ed2a6acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:33.641929 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a52a7-c20f-4f1b-af51-d331f82d55fe service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:33.642211 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a52a7-c20f-4f1b-af51-d331f82d55fe service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:33.642487 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a52a7-c20f-4f1b-af51-d331f82d55fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:33.652668 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a52a7-c20f-4f1b-af51-d331f82d55fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:33.654259 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a52a7-c20f-4f1b-af51-d331f82d55fe service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:33.666147 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1462/2945] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:42.384263 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=6 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:43.660697 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=6 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:46.451795 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be550e30-9f83-48b0-afe0-2b416bc7993b service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:46.460718 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be550e30-9f83-48b0-afe0-2b416bc7993b service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:46.464317 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d163f-7e88-44e5-a79c-442949854ed8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db814952f9f466abf6148b7a0db2173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:46.464907 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d163f-7e88-44e5-a79c-442949854ed8 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:46.465186 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d163f-7e88-44e5-a79c-442949854ed8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:46.465582 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d163f-7e88-44e5-a79c-442949854ed8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:46.534783 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d163f-7e88-44e5-a79c-442949854ed8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:46.536270 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d163f-7e88-44e5-a79c-442949854ed8 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:46.555433 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1484/2946] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:49.709766 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a52a7-c20f-4f1b-af51-d331f82d55fe service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:49.722147 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a52a7-c20f-4f1b-af51-d331f82d55fe service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:49.727356 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca167026-624b-4c5b-9bff-c7b843406981 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff152e22374407eb72caf648a763107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:49.727988 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca167026-624b-4c5b-9bff-c7b843406981 service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:49.728272 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca167026-624b-4c5b-9bff-c7b843406981 service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:49.728544 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca167026-624b-4c5b-9bff-c7b843406981 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:49.730319 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ca167026-624b-4c5b-9bff-c7b843406981 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:49.742348 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca167026-624b-4c5b-9bff-c7b843406981 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:49.743963 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca167026-624b-4c5b-9bff-c7b843406981 service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:49.755229 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1463/2947] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:53.251271 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d163f-7e88-44e5-a79c-442949854ed8 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:53.267486 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d163f-7e88-44e5-a79c-442949854ed8 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:53.273892 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b159b-fbbf-46fb-8107-c32f01210148 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff152e22374407eb72caf648a763107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:53.274786 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b159b-fbbf-46fb-8107-c32f01210148 service placement] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:53.275314 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b159b-fbbf-46fb-8107-c32f01210148 service placement] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:53.275668 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b159b-fbbf-46fb-8107-c32f01210148 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:53.278325 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ee0b159b-fbbf-46fb-8107-c32f01210148 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:53.292167 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b159b-fbbf-46fb-8107-c32f01210148 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:53.294836 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b159b-fbbf-46fb-8107-c32f01210148 service placement] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:53.305783 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1485/2948] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:30:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:30:53.381638 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca167026-624b-4c5b-9bff-c7b843406981 service placement] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:53.393167 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca167026-624b-4c5b-9bff-c7b843406981 service placement] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:53.397841 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c853babc-dbc3-4263-b1a4-6ef0fcbd0bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:53.398465 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c853babc-dbc3-4263-b1a4-6ef0fcbd0bf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:53.398745 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c853babc-dbc3-4263-b1a4-6ef0fcbd0bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:53.399036 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c853babc-dbc3-4263-b1a4-6ef0fcbd0bf0 admin admin] PATH_INFO: `/v3/users/eaa1bdd8c2f34ebc90d7f5b6ec4c2d51` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:53.400168 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c853babc-dbc3-4263-b1a4-6ef0fcbd0bf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaa1bdd8c2f34ebc90d7f5b6ec4c2d51)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:53.402071 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c853babc-dbc3-4263-b1a4-6ef0fcbd0bf0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:53.616792 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c853babc-dbc3-4263-b1a4-6ef0fcbd0bf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaa1bdd8c2f34ebc90d7f5b6ec4c2d51'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:53.671941 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c853babc-dbc3-4263-b1a4-6ef0fcbd0bf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaa1bdd8c2f34ebc90d7f5b6ec4c2d51'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:53.672600 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c853babc-dbc3-4263-b1a4-6ef0fcbd0bf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaa1bdd8c2f34ebc90d7f5b6ec4c2d51'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:53.673227 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c853babc-dbc3-4263-b1a4-6ef0fcbd0bf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaa1bdd8c2f34ebc90d7f5b6ec4c2d51'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:53.680609 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c853babc-dbc3-4263-b1a4-6ef0fcbd0bf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaa1bdd8c2f34ebc90d7f5b6ec4c2d51'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:53.697548 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1464/2949] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:30:53 2022] DELETE /identity/v3/users/eaa1bdd8c2f34ebc90d7f5b6ec4c2d51 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:30:53.971796 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b159b-fbbf-46fb-8107-c32f01210148 service placement] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:53.983935 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b159b-fbbf-46fb-8107-c32f01210148 service placement] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:54.041388 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193055ca-34fd-4f4e-8fa6-bde44319cf27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:54.042017 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193055ca-34fd-4f4e-8fa6-bde44319cf27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:54.042299 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193055ca-34fd-4f4e-8fa6-bde44319cf27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:54.042575 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193055ca-34fd-4f4e-8fa6-bde44319cf27 admin admin] PATH_INFO: `/v3/projects/2a40aefbd17f4dbc911527a917f03976` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:54.044359 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193055ca-34fd-4f4e-8fa6-bde44319cf27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a40aefbd17f4dbc911527a917f03976)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:54.046260 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193055ca-34fd-4f4e-8fa6-bde44319cf27 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:54.159550 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-193055ca-34fd-4f4e-8fa6-bde44319cf27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a40aefbd17f4dbc911527a917f03976', 'request_id': 'req-193055ca-34fd-4f4e-8fa6-bde44319cf27'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:54.165104 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-193055ca-34fd-4f4e-8fa6-bde44319cf27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a40aefbd17f4dbc911527a917f03976', 'request_id': 'req-193055ca-34fd-4f4e-8fa6-bde44319cf27'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:54.232441 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-193055ca-34fd-4f4e-8fa6-bde44319cf27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a40aefbd17f4dbc911527a917f03976', 'request_id': 'req-193055ca-34fd-4f4e-8fa6-bde44319cf27'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:54.233582 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:54.257581 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-193055ca-34fd-4f4e-8fa6-bde44319cf27 admin admin] The token cache is being invalidate because project 2a40aefbd17f4dbc911527a917f03976 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:30:54.258196 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-193055ca-34fd-4f4e-8fa6-bde44319cf27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:54.258673 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-193055ca-34fd-4f4e-8fa6-bde44319cf27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:54.260274 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1486/2950] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:30:53 2022] DELETE /identity/v3/projects/2a40aefbd17f4dbc911527a917f03976 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:30:54.265160 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c853babc-dbc3-4263-b1a4-6ef0fcbd0bf0 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:54.266663 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c853babc-dbc3-4263-b1a4-6ef0fcbd0bf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:54.281363 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c853babc-dbc3-4263-b1a4-6ef0fcbd0bf0 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:54.338507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537027d2-b9cf-4d82-9af4-b0c214ada06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:54.339139 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537027d2-b9cf-4d82-9af4-b0c214ada06c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:54.339476 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537027d2-b9cf-4d82-9af4-b0c214ada06c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:54.339740 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537027d2-b9cf-4d82-9af4-b0c214ada06c admin admin] PATH_INFO: `/v3/users/d90e642fcfbc4664988badec25fb8e4a` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:54.341507 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537027d2-b9cf-4d82-9af4-b0c214ada06c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d90e642fcfbc4664988badec25fb8e4a)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:54.343444 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537027d2-b9cf-4d82-9af4-b0c214ada06c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:54.532326 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-537027d2-b9cf-4d82-9af4-b0c214ada06c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd90e642fcfbc4664988badec25fb8e4a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:54.609268 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-537027d2-b9cf-4d82-9af4-b0c214ada06c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd90e642fcfbc4664988badec25fb8e4a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:54.609887 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-537027d2-b9cf-4d82-9af4-b0c214ada06c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd90e642fcfbc4664988badec25fb8e4a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:54.610549 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-537027d2-b9cf-4d82-9af4-b0c214ada06c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd90e642fcfbc4664988badec25fb8e4a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:54.618084 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:30:54.620923 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-537027d2-b9cf-4d82-9af4-b0c214ada06c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd90e642fcfbc4664988badec25fb8e4a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:54.638955 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1465/2951] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 13:30:54 2022] DELETE /identity/v3/users/d90e642fcfbc4664988badec25fb8e4a => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:30:54.879537 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193055ca-34fd-4f4e-8fa6-bde44319cf27 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:30:54.880996 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-193055ca-34fd-4f4e-8fa6-bde44319cf27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:30:54.896229 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193055ca-34fd-4f4e-8fa6-bde44319cf27 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:30:54.954051 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323db2eb-cfba-4811-a8b4-45e4b5ba2d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '903d09eed7334b43af767deea1a6bd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c65e78604254a1dbcd377669ee26423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:30:54.954676 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323db2eb-cfba-4811-a8b4-45e4b5ba2d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:30:54.954970 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323db2eb-cfba-4811-a8b4-45e4b5ba2d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:30:54.955253 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323db2eb-cfba-4811-a8b4-45e4b5ba2d1f admin admin] PATH_INFO: `/v3/projects/a17be39fe9754ac593beda82dd4a1e54` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:30:54.956353 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323db2eb-cfba-4811-a8b4-45e4b5ba2d1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a17be39fe9754ac593beda82dd4a1e54)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:30:54.958242 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323db2eb-cfba-4811-a8b4-45e4b5ba2d1f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:30:55.084037 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-323db2eb-cfba-4811-a8b4-45e4b5ba2d1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a17be39fe9754ac593beda82dd4a1e54', 'request_id': 'req-323db2eb-cfba-4811-a8b4-45e4b5ba2d1f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:55.089434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-323db2eb-cfba-4811-a8b4-45e4b5ba2d1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a17be39fe9754ac593beda82dd4a1e54', 'request_id': 'req-323db2eb-cfba-4811-a8b4-45e4b5ba2d1f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:55.162590 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-323db2eb-cfba-4811-a8b4-45e4b5ba2d1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a17be39fe9754ac593beda82dd4a1e54', 'request_id': 'req-323db2eb-cfba-4811-a8b4-45e4b5ba2d1f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:55.174985 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-323db2eb-cfba-4811-a8b4-45e4b5ba2d1f admin admin] The token cache is being invalidate because project a17be39fe9754ac593beda82dd4a1e54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 13:30:55.175318 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-323db2eb-cfba-4811-a8b4-45e4b5ba2d1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:55.175949 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-323db2eb-cfba-4811-a8b4-45e4b5ba2d1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 13:30:55.177434 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1487/2952] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 13:30:54 2022] DELETE /identity/v3/projects/a17be39fe9754ac593beda82dd4a1e54 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 13:31:04.781364 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:05.171950 dv-cinder-851996-7-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=59,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 13:31:14.718399 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537027d2-b9cf-4d82-9af4-b0c214ada06c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 13:31:14.719841 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-537027d2-b9cf-4d82-9af4-b0c214ada06c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:14.810333 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537027d2-b9cf-4d82-9af4-b0c214ada06c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 13:31:14.869226 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b920a1-1203-459c-98c7-a652c622d4d5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ff152e22374407eb72caf648a763107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e201e5f281a4977be503e22daf149d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 13:31:14.869830 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b920a1-1203-459c-98c7-a652c622d4d5 service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 13:31:14.870109 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b920a1-1203-459c-98c7-a652c622d4d5 service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 13:31:14.870382 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b920a1-1203-459c-98c7-a652c622d4d5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 13:31:14.872091 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e4b920a1-1203-459c-98c7-a652c622d4d5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 13:31:14.883008 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b920a1-1203-459c-98c7-a652c622d4d5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 13:31:14.884615 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b920a1-1203-459c-98c7-a652c622d4d5 service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 13:31:14.894401 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1466/2953] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 13:31:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 13:31:24.890168 dv-cinder-851996-7-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=67 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}